[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:11:40 dist-test-slave-h3ft krb5kdc[9758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:11:40 dist-test-slave-h3ft krb5kdc[9758](info): set up 2 sockets
Aug 07 06:11:40 dist-test-slave-h3ft krb5kdc[9758](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 06:11:42.290683 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.432s user 0.011s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:11:42 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:11:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:11:42.443120 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:43105
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:43105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:42.616533 9775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:42.616935 9775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:42.617022 9775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:42.621420 9775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:11:42.621527 9775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:42.621563 9775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:42.621603 9775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:11:42.621642 9775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:11:42.628278 9775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:43105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:43105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:42.630942 9775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:42.633347 9775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:42.651389 9782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:42.651822 9786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:42.653509 9781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:42.654809 9775 server_base.cc:1034] running on GCE node
I20260807 06:11:42.655880 9775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:42.658134 9775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:42.659487 9775 hybrid_clock.cc:648] HybridClock initialized: now 1786083102659453 us; error 57 us; skew 500 ppm
Aug 07 06:11:42 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083102, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:42.667102 9775 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:11:42.668740 9775 webserver.cc:460] Webserver started at http://127.9.133.190:34009/ using document root <none> and password file <none>
I20260807 06:11:42.669857 9775 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:42.669986 9775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:42.670469 9775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:42.674053 9775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "4b3080bca83e4b7692ea4dedf8bd339c"
format_stamp: "Formatted at 2026-08-07 06:11:42 on dist-test-slave-h3ft"
server_key: "ace9feb304c32b04fe5313c793f1c69e"
server_key_iv: "499178cfeafe571fb0ce03814eb8e055"
server_key_version: "encryptionkey@0"
I20260807 06:11:42.675097 9775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "4b3080bca83e4b7692ea4dedf8bd339c"
format_stamp: "Formatted at 2026-08-07 06:11:42 on dist-test-slave-h3ft"
server_key: "ace9feb304c32b04fe5313c793f1c69e"
server_key_iv: "499178cfeafe571fb0ce03814eb8e055"
server_key_version: "encryptionkey@0"
I20260807 06:11:42.681630 9775 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 06:11:42.686240 9796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:42.688540 9775 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 06:11:42.688836 9775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "4b3080bca83e4b7692ea4dedf8bd339c"
format_stamp: "Formatted at 2026-08-07 06:11:42 on dist-test-slave-h3ft"
server_key: "ace9feb304c32b04fe5313c793f1c69e"
server_key_iv: "499178cfeafe571fb0ce03814eb8e055"
server_key_version: "encryptionkey@0"
I20260807 06:11:42.689043 9775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:42.746488 9775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:42.751240 9775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:42.751578 9775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:42.762359 9775 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:43105
I20260807 06:11:42.762392 9869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:43105 every 8 connection(s)
I20260807 06:11:42.763935 9775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:11:42.766224 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9775
I20260807 06:11:42.766451 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:11:42.768188 9750 external_mini_cluster.cc:1428] Setting key 86c3d4992ee9012ed47939edb9dbecb4
I20260807 06:11:42.769893 9871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:42.781837 9871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c: Bootstrap starting.
I20260807 06:11:42.787509 9871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:42.789137 9871 log.cc:824] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:42.793587 9871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c: No bootstrap required, opened a new log
Aug 07 06:11:42 dist-test-slave-h3ft krb5kdc[9758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:42.799634 9871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 43105 } }
I20260807 06:11:42.800076 9871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:42.800204 9871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3080bca83e4b7692ea4dedf8bd339c, State: Initialized, Role: FOLLOWER
I20260807 06:11:42.801270 9871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 43105 } }
I20260807 06:11:42.801534 9871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:11:42.801721 9871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:11:42.801909 9871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:42.803572 9871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 43105 } }
I20260807 06:11:42.804464 9871 leader_election.cc:302] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b3080bca83e4b7692ea4dedf8bd339c; no voters:
I20260807 06:11:42.805058 9871 leader_election.cc:288] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:11:42.805259 9876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:42.805359 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:42.775681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.9.133.190:43105)
0807 06:11:42.776164 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:42.776180 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:42.776226 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:11:42.781482 (+ 5256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:42.781512 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:42.782148 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:42.782504 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:42.786767 (+ 4263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:42.788090 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:42.790201 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:42.790940 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:42.791213 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:42.797531 (+ 6318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:42.797574 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:42.797591 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:42.797636 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:42.799929 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:42.800951 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:42.800968 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:42.800973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:42.801136 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:42.801630 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:42.801634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:42.801636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:42.802447 (+ 811us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:42.802637 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:42.803574 (+ 937us) server_negotiation.cc:299] Negotiation successful
0807 06:11:42.803944 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":175,"threads_started":1}
I20260807 06:11:42.805508 9876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [term 1 LEADER]: Becoming Leader. State: Replica: 4b3080bca83e4b7692ea4dedf8bd339c, State: Running, Role: LEADER
I20260807 06:11:42.806150 9876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 43105 } }
I20260807 06:11:42.807130 9871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:11:42.809234 9878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b3080bca83e4b7692ea4dedf8bd339c. Latest consensus state: current_term: 1 leader_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 43105 } } }
I20260807 06:11:42.809386 9878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:42.809947 9877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3080bca83e4b7692ea4dedf8bd339c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 43105 } } }
I20260807 06:11:42.810078 9877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c [sys.catalog]: This master's current role is: LEADER
W20260807 06:11:42.815529 9894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:11:42.815692 9894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:11:42.815790 9888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:11:42.816816 9888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:11:42.827112 9888 catalog_manager.cc:1347] Generated new cluster ID: b4b37bb795c24a30915e81905e7addd5
I20260807 06:11:42.827239 9888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:11:42.857376 9888 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:11:42.858763 9888 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:11:42.873015 9888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b3080bca83e4b7692ea4dedf8bd339c: Generated new TSK 0
I20260807 06:11:42.874220 9888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:11:42.920917 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:43105
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:43.061631 9906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:43.061969 9906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:43.062021 9906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:43.066118 9906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:43.066191 9906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:43.066253 9906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:11:43.072094 9906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:43105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:43.073964 9906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:43.075363 9906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:43.087860 9912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:43.088018 9906 server_base.cc:1034] running on GCE node
W20260807 06:11:43.087916 9916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:43.087868 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:43.088675 9906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:43.089499 9906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:43.090670 9906 hybrid_clock.cc:648] HybridClock initialized: now 1786083103090638 us; error 47 us; skew 500 ppm
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:43.094309 9906 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:11:43.095510 9906 webserver.cc:460] Webserver started at http://127.9.133.129:38583/ using document root <none> and password file <none>
I20260807 06:11:43.096231 9906 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:43.096302 9906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:43.096573 9906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:43.099304 9906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "34069fba000c4ae9945531c9e60ed49d"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "504fcd2dbd8836c99259101172fb5254"
server_key_iv: "15c9f6ee10448b5d28533a3e955c2cbe"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.100028 9906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "34069fba000c4ae9945531c9e60ed49d"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "504fcd2dbd8836c99259101172fb5254"
server_key_iv: "15c9f6ee10448b5d28533a3e955c2cbe"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.105003 9906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:11:43.108333 9925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.110167 9906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:11:43.110368 9906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "34069fba000c4ae9945531c9e60ed49d"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "504fcd2dbd8836c99259101172fb5254"
server_key_iv: "15c9f6ee10448b5d28533a3e955c2cbe"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.110512 9906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:43.129695 9906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:43.133977 9906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:43.134193 9906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:43.135092 9906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:11:43.136520 9906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:43.136580 9906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.136631 9906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:43.136646 9906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.148444 9906 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:37163
I20260807 06:11:43.148458 10080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:37163 every 8 connection(s)
I20260807 06:11:43.149632 9906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:11:43.150498 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9906
I20260807 06:11:43.150668 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:11:43.151165 9750 external_mini_cluster.cc:1428] Setting key 7a65e70797a21ce3b8733a3b58d1787e
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:11:43.169052 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.152710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:38633 (local address 127.9.133.190:43105)
0807 06:11:43.153156 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:43.153160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:43.154323 (+ 1163us) server_negotiation.cc:407] Connection header received
0807 06:11:43.156924 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:43.156930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:43.157013 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:43.157111 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:43.158527 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.159219 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.161209 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.161470 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.161569 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.165055 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:43.165059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:43.165064 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:43.165101 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:43.166751 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.167394 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.167397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.167398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.167497 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.167811 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.167813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.167814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.168006 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:43.168082 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:43.168686 (+ 604us) server_negotiation.cc:299] Negotiation successful
0807 06:11:43.168806 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346}
I20260807 06:11:43.170120 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.153350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:43105 (local address 127.9.133.129:38633)
0807 06:11:43.154103 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:43.154175 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:43.156678 (+ 2503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:43.157313 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:43.157321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:43.157734 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:43.158366 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:43.158381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.159359 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.159365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:43.161038 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:43.161051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.161666 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.161673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:43.161937 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.162708 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:43.162738 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:43.164840 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:43.166859 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:43.166864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:43.166877 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:43.167316 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:43.167577 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:43.167580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:43.167581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:43.167726 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:43.168090 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:43.168095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:43.168457 (+ 362us) client_negotiation.cc:769] Sending connection context
0807 06:11:43.168627 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 06:11:43.169016 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":141,"threads_started":1}
I20260807 06:11:43.171860 10081 heartbeater.cc:347] Connected to a master server at 127.9.133.190:43105
I20260807 06:11:43.172179 10081 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:43.172854 10081 heartbeater.cc:511] Master 127.9.133.190:43105 requested a full tablet report, sending...
I20260807 06:11:43.175485 9818 ts_manager.cc:194] Registered new tserver with Master: 34069fba000c4ae9945531c9e60ed49d (127.9.133.129:37163)
I20260807 06:11:43.177100 9818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:38633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:11:43.188380 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:43105
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:43.321971 10086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:43.322291 10086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:43.322340 10086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:43.326524 10086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:43.326593 10086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:43.326658 10086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:11:43.332429 10086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:43105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:43.333818 10086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:43.335093 10086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:43.347203 10094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:43.347358 10086 server_base.cc:1034] running on GCE node
W20260807 06:11:43.347206 10097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:43.347203 10095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:43.348008 10086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:43.348775 10086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:43.349951 10086 hybrid_clock.cc:648] HybridClock initialized: now 1786083103349909 us; error 50 us; skew 500 ppm
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:43.353176 10086 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:11:43.354236 10086 webserver.cc:460] Webserver started at http://127.9.133.130:41015/ using document root <none> and password file <none>
I20260807 06:11:43.354923 10086 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:43.354993 10086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:43.355341 10086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:43.357769 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "841309ae8fcb47e093f44fc450604c56"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "5eb543d150f758f31532ce361d5e3aa7"
server_key_iv: "8fbb29c3214e8f358647f8aea8756279"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.358400 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "841309ae8fcb47e093f44fc450604c56"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "5eb543d150f758f31532ce361d5e3aa7"
server_key_iv: "8fbb29c3214e8f358647f8aea8756279"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.362823 10086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 06:11:43.365860 10105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.367411 10086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:11:43.367556 10086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "841309ae8fcb47e093f44fc450604c56"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "5eb543d150f758f31532ce361d5e3aa7"
server_key_iv: "8fbb29c3214e8f358647f8aea8756279"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.367669 10086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:43.387531 10086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:43.390945 10086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:43.391252 10086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:43.392040 10086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:11:43.393122 10086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:43.393179 10086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.393268 10086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:43.393307 10086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.405473 10086 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:45459
I20260807 06:11:43.405496 10258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:45459 every 8 connection(s)
I20260807 06:11:43.406677 10086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:11:43.406965 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10086
I20260807 06:11:43.407107 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:11:43.407450 9750 external_mini_cluster.cc:1428] Setting key 749f69fb7add72d93f18e41c3774108d
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:11:43.424839 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.409006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:38597 (local address 127.9.133.190:43105)
0807 06:11:43.409164 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:43.409168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:43.410504 (+ 1336us) server_negotiation.cc:407] Connection header received
0807 06:11:43.412777 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:43.412781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:43.412844 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:43.412940 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:43.414505 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.415224 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.417219 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.417458 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.417545 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.420832 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:43.420835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:43.420838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:43.420864 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:43.422381 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.422935 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.422939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.422941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.423075 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.423448 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.423452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.423454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.423700 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:43.423814 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:43.424479 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 06:11:43.424631 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:11:43.425843 10263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.409645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:43105 (local address 127.9.133.130:38597)
0807 06:11:43.410293 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:43.410358 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:43.412489 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:43.413140 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:43.413148 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:43.413570 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:43.414300 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:43.414326 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.415358 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.415364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:43.417089 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:43.417103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.417632 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.417638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:43.417899 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.418678 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:43.418703 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:43.420646 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:43.422516 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:43.422522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:43.422539 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:43.422816 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:43.423165 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:43.423168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:43.423170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:43.423349 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:43.423813 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:43.423822 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:43.424188 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 06:11:43.424428 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 06:11:43.424804 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":188,"threads_started":1}
I20260807 06:11:43.427290 10259 heartbeater.cc:347] Connected to a master server at 127.9.133.190:43105
I20260807 06:11:43.427625 10259 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:43.428344 10259 heartbeater.cc:511] Master 127.9.133.190:43105 requested a full tablet report, sending...
I20260807 06:11:43.429694 9818 ts_manager.cc:194] Registered new tserver with Master: 841309ae8fcb47e093f44fc450604c56 (127.9.133.130:45459)
I20260807 06:11:43.430373 9818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:38597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:11:43.442514 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:43105
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:43.579569 10270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:43.579995 10270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:43.580070 10270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:43.584407 10270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:43.584477 10270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:43.584573 10270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:11:43.590453 10270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:43105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:43.591897 10270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:43.593201 10270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:43.604568 10277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:43.604590 10279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:43.604758 10270 server_base.cc:1034] running on GCE node
W20260807 06:11:43.604686 10283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:43.605432 10270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:43.606204 10270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:43.607378 10270 hybrid_clock.cc:648] HybridClock initialized: now 1786083103607356 us; error 33 us; skew 500 ppm
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:43.610838 10270 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:11:43.612004 10270 webserver.cc:460] Webserver started at http://127.9.133.131:46299/ using document root <none> and password file <none>
I20260807 06:11:43.612761 10270 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:43.612831 10270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:43.613094 10270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:43.615753 10270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "e2ef7dc7d28142bd94b63c3a46792933"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "bc53c4aa0e9c0f552981a5f075b02661"
server_key_iv: "7ed40c10831e648206d5b4b31b8b9663"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.616405 10270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "e2ef7dc7d28142bd94b63c3a46792933"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "bc53c4aa0e9c0f552981a5f075b02661"
server_key_iv: "7ed40c10831e648206d5b4b31b8b9663"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.620859 10270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:11:43.623659 10290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.625149 10270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:11:43.625275 10270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "e2ef7dc7d28142bd94b63c3a46792933"
format_stamp: "Formatted at 2026-08-07 06:11:43 on dist-test-slave-h3ft"
server_key: "bc53c4aa0e9c0f552981a5f075b02661"
server_key_iv: "7ed40c10831e648206d5b4b31b8b9663"
server_key_version: "encryptionkey@0"
I20260807 06:11:43.625383 10270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:43.637351 10270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:43.640761 10270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:43.641005 10270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:43.641682 10270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:11:43.642714 10270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:43.642781 10270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.642849 10270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:43.642891 10270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:43.654726 10270 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:41421
I20260807 06:11:43.654753 10445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:41421 every 8 connection(s)
I20260807 06:11:43.655856 10270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:11:43.661548 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10270
I20260807 06:11:43.661721 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:11:43.662106 9750 external_mini_cluster.cc:1428] Setting key 9679ee8024b6257f03ab8fda5f9a0c4b
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:43.672544 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.658135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:60819 (local address 127.9.133.190:43105)
0807 06:11:43.658332 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:43.658337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:43.659422 (+ 1085us) server_negotiation.cc:407] Connection header received
0807 06:11:43.661032 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:43.661036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:43.661095 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:43.661178 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:43.663145 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.663961 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.665486 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.665736 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.665823 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.668825 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:43.668828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:43.668830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:43.668855 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:43.670310 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.670835 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.670838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.670839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.670921 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.671300 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.671303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.671304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.671512 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:43.671591 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:43.672241 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 06:11:43.672383 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:11:43.673521 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.658615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:43105 (local address 127.9.133.131:60819)
0807 06:11:43.659247 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:43.659298 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:43.660794 (+ 1496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:43.661447 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:43.661458 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:43.662114 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:43.662932 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:43.662960 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.664110 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.664114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:43.665357 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:43.665368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.665875 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.665878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:43.666084 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.666819 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:43.666839 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:43.668652 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:43.670417 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:43.670422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:43.670436 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:43.670718 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:43.671042 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:43.671051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:43.671053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:43.671207 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:43.671603 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:43.671625 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:43.671955 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 06:11:43.672138 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 06:11:43.672441 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":131,"threads_started":1}
I20260807 06:11:43.674916 10449 heartbeater.cc:347] Connected to a master server at 127.9.133.190:43105
I20260807 06:11:43.675228 10449 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:43.675902 10449 heartbeater.cc:511] Master 127.9.133.190:43105 requested a full tablet report, sending...
I20260807 06:11:43.677286 9817 ts_manager.cc:194] Registered new tserver with Master: e2ef7dc7d28142bd94b63c3a46792933 (127.9.133.131:41421)
I20260807 06:11:43.677979 9817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:60819
I20260807 06:11:43.689353 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:11:43.723383 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.712105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.9.133.129:37163)
0807 06:11:43.712510 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:43.712518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:43.712556 (+ 38us) server_negotiation.cc:407] Connection header received
0807 06:11:43.712659 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:43.712666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:43.712943 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:43.713106 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:43.713875 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.714852 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.715979 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.716454 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.716585 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.719520 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:43.719531 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:43.719549 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:43.719580 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:43.721381 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.721832 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.721839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.721840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.721973 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.722209 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.722211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.722212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.722513 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:43.722718 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:43.723039 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 06:11:43.723209 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":90,"threads_started":1}
W20260807 06:11:43.724798 9965 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55202
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[9758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:11:43.757073 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:43.747764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.9.133.129:37163)
0807 06:11:43.747964 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:43.747969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:43.748018 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:11:43.748179 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:43.748182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:43.748241 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:43.748322 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:43.749102 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.749680 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.750517 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:43.750745 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:43.750826 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:43.753603 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:43.753606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:43.753608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:43.753634 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:43.755200 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.755662 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.755665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.755667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.755761 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:43.756102 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:43.756106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:43.756108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:43.756372 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:43.756512 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:43.756780 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 06:11:43.756907 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 06:11:43.758551 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9906
I20260807 06:11:43.776585 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10086
I20260807 06:11:43.790107 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10270
I20260807 06:11:43.802811 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9775
2026-08-07T06:11:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2996 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[10468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[10468](info): set up 2 sockets
Aug 07 06:11:43 dist-test-slave-h3ft krb5kdc[10468](info): commencing operation
krb5kdc: starting...
W20260807 06:11:45.067826 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:11:45 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:11:45.222508 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:38841
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:38841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:45.359633 10490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:45.360019 10490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:45.360105 10490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:45.364204 10490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:11:45.364302 10490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:45.364338 10490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:45.364385 10490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:11:45.364439 10490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:11:45.370028 10490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:38841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:38841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:45.371429 10490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:45.372716 10490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:45.383222 10499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:45.383222 10498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:45.383389 10490 server_base.cc:1034] running on GCE node
W20260807 06:11:45.383267 10502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:45.384228 10490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:45.385545 10490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:45.386745 10490 hybrid_clock.cc:648] HybridClock initialized: now 1786083105386717 us; error 40 us; skew 500 ppm
Aug 07 06:11:45 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:45.390233 10490 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:11:45.391353 10490 webserver.cc:460] Webserver started at http://127.9.133.190:40217/ using document root <none> and password file <none>
I20260807 06:11:45.392077 10490 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:45.392170 10490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:45.392434 10490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:45.394815 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "5a82bed408f14310855487228467cb95"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "783c38d83f9acd2c955399a83fe5ffe4"
server_key_iv: "cde4512b40bb371f9de1532c5af7d2ae"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.395512 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "5a82bed408f14310855487228467cb95"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "783c38d83f9acd2c955399a83fe5ffe4"
server_key_iv: "cde4512b40bb371f9de1532c5af7d2ae"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.399777 10490 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:11:45.402606 10510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:45.404189 10490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 06:11:45.404351 10490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "5a82bed408f14310855487228467cb95"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "783c38d83f9acd2c955399a83fe5ffe4"
server_key_iv: "cde4512b40bb371f9de1532c5af7d2ae"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.404481 10490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:45.417634 10490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:45.421409 10490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:45.421638 10490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:45.430094 10490 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:38841
I20260807 06:11:45.430155 10577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:38841 every 8 connection(s)
I20260807 06:11:45.431336 10490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:11:45.432390 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10490
I20260807 06:11:45.432533 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:11:45.432852 9750 external_mini_cluster.cc:1428] Setting key 521612f215b0e706bf79b38215cfd5ce
I20260807 06:11:45.435352 10579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 06:11:45 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:45.443614 10579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95: Bootstrap starting.
I20260807 06:11:45.447252 10579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:45.448233 10579 log.cc:824] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:45.450311 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:45.434508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.9.133.190:38841)
0807 06:11:45.435050 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:45.435075 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:45.435120 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:11:45.435930 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:45.435955 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:45.436348 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:45.436694 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:45.437761 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:45.438915 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:45.440232 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:45.440814 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:45.441051 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:45.444310 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:45.444319 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:45.444338 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:45.444386 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:45.446779 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:45.447435 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:45.447442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:45.447448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:45.447590 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:45.447922 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:45.447926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:45.447927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:45.448488 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:45.448715 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:45.449057 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 06:11:45.449426 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":166,"threads_started":1}
I20260807 06:11:45.450732 10579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95: No bootstrap required, opened a new log
I20260807 06:11:45.455698 10579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a82bed408f14310855487228467cb95" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38841 } }
I20260807 06:11:45.456002 10579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:45.456086 10579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a82bed408f14310855487228467cb95, State: Initialized, Role: FOLLOWER
I20260807 06:11:45.456612 10579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a82bed408f14310855487228467cb95" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38841 } }
I20260807 06:11:45.456786 10579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:11:45.456881 10579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:11:45.457011 10579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:45.458196 10579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a82bed408f14310855487228467cb95" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38841 } }
I20260807 06:11:45.458621 10579 leader_election.cc:302] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a82bed408f14310855487228467cb95; no voters:
I20260807 06:11:45.459156 10579 leader_election.cc:288] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:11:45.459244 10586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:45.459468 10586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [term 1 LEADER]: Becoming Leader. State: Replica: 5a82bed408f14310855487228467cb95, State: Running, Role: LEADER
I20260807 06:11:45.459813 10586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a82bed408f14310855487228467cb95" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38841 } }
I20260807 06:11:45.460772 10579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:11:45.461923 10587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a82bed408f14310855487228467cb95. Latest consensus state: current_term: 1 leader_uuid: "5a82bed408f14310855487228467cb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a82bed408f14310855487228467cb95" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38841 } } }
I20260807 06:11:45.461948 10590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a82bed408f14310855487228467cb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a82bed408f14310855487228467cb95" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38841 } } }
I20260807 06:11:45.462116 10590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:45.462117 10587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95 [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:45.462632 10598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:11:45.466653 10598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:11:45.473783 10598 catalog_manager.cc:1347] Generated new cluster ID: 729bb7a00e59468baeaa5c2827526a7d
I20260807 06:11:45.473877 10598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:11:45.482774 10598 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:11:45.484086 10598 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:11:45.492861 10598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a82bed408f14310855487228467cb95: Generated new TSK 0
I20260807 06:11:45.493850 10598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:11:45.535382 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:38841
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:45.690956 10613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:45.691461 10613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:45.691517 10613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:45.695986 10613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:45.696086 10613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:45.696162 10613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:11:45.702409 10613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:38841
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:45.704049 10613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:45.705470 10613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:45.717955 10621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:45.717955 10623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:45.718034 10619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:45.718156 10613 server_base.cc:1034] running on GCE node
I20260807 06:11:45.718878 10613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:45.719779 10613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:45.720980 10613 hybrid_clock.cc:648] HybridClock initialized: now 1786083105720955 us; error 38 us; skew 500 ppm
Aug 07 06:11:45 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:45.724740 10613 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:11:45.725901 10613 webserver.cc:460] Webserver started at http://127.9.133.129:35053/ using document root <none> and password file <none>
I20260807 06:11:45.726629 10613 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:45.726699 10613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:45.726980 10613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:45.729665 10613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "8f40d4f7622147fe85ac6fd99c35d18f"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "1254ec379a7c9514bda038eaa68a0b14"
server_key_iv: "c55facb092be9118e97e56d31a70eca0"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.730401 10613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "8f40d4f7622147fe85ac6fd99c35d18f"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "1254ec379a7c9514bda038eaa68a0b14"
server_key_iv: "c55facb092be9118e97e56d31a70eca0"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.735430 10613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 06:11:45.738843 10632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:45.740713 10613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:11:45.740860 10613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "8f40d4f7622147fe85ac6fd99c35d18f"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "1254ec379a7c9514bda038eaa68a0b14"
server_key_iv: "c55facb092be9118e97e56d31a70eca0"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.741182 10613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:45.756683 10613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:45.760697 10613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:45.760951 10613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:45.761823 10613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:11:45.762930 10613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:45.762991 10613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:45.763096 10613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:45.763119 10613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:45.773703 10613 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:43889
I20260807 06:11:45.773726 10798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:43889 every 8 connection(s)
I20260807 06:11:45.774881 10613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:11:45.776523 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10613
I20260807 06:11:45.776674 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:11:45.776999 9750 external_mini_cluster.cc:1428] Setting key 387ec61db056bf3e978a12c08ca0213e
Aug 07 06:11:45 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:11:45.794068 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:45.777423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:59887 (local address 127.9.133.190:38841)
0807 06:11:45.777611 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:45.777615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:45.778788 (+ 1173us) server_negotiation.cc:407] Connection header received
0807 06:11:45.780359 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:45.780363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:45.780418 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:45.780500 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:45.782387 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:45.783364 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:45.784656 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:45.785110 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:45.785243 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:45.788560 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:45.788567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:45.788571 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:45.788607 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:45.790797 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:45.791637 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:45.791640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:45.791642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:45.791744 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:45.792339 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:45.792342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:45.792344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:45.792637 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:45.792728 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:45.793570 (+ 842us) server_negotiation.cc:299] Negotiation successful
0807 06:11:45.793791 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 06:11:45.795225 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:45.777950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:38841 (local address 127.9.133.129:59887)
0807 06:11:45.778575 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:45.778651 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:45.780055 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:45.780757 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:45.780769 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:45.781432 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:45.782198 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:45.782216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:45.783510 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:45.783514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:45.784512 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:45.784522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:45.785357 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:45.785363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:45.785570 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:45.786331 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:45.786350 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:45.788342 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:45.791140 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:45.791147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:45.791167 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:45.791507 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:45.792017 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:45.792020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:45.792022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:45.792204 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:45.792766 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:45.792773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:45.793248 (+ 475us) client_negotiation.cc:769] Sending connection context
0807 06:11:45.793462 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 06:11:45.793913 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":128,"threads_started":1}
I20260807 06:11:45.796876 10800 heartbeater.cc:347] Connected to a master server at 127.9.133.190:38841
I20260807 06:11:45.797233 10800 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:45.798058 10800 heartbeater.cc:511] Master 127.9.133.190:38841 requested a full tablet report, sending...
I20260807 06:11:45.800191 10534 ts_manager.cc:194] Registered new tserver with Master: 8f40d4f7622147fe85ac6fd99c35d18f (127.9.133.129:43889)
I20260807 06:11:45.801597 10534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:59887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:11:45.814872 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:38841
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:45.950112 10805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:45.950441 10805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:45.950492 10805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:45.954758 10805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:45.954838 10805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:45.954902 10805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:11:45.960611 10805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:38841
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:45.962157 10805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:45.963732 10805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:45.975355 10815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:45.975379 10819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:45.975379 10816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:45.975637 10805 server_base.cc:1034] running on GCE node
I20260807 06:11:45.976276 10805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:45.977051 10805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:45.978241 10805 hybrid_clock.cc:648] HybridClock initialized: now 1786083105978216 us; error 31 us; skew 500 ppm
Aug 07 06:11:45 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:45.981606 10805 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:11:45.982882 10805 webserver.cc:460] Webserver started at http://127.9.133.130:38285/ using document root <none> and password file <none>
I20260807 06:11:45.983852 10805 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:45.983950 10805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:45.984218 10805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:45.986680 10805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "b9a5e83955a344439ef076585e0370c2"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "68462a22e9b91ef763b903c0d0faa27d"
server_key_iv: "44f0496a1de8b523d632c37691de8f61"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.987355 10805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "b9a5e83955a344439ef076585e0370c2"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "68462a22e9b91ef763b903c0d0faa27d"
server_key_iv: "44f0496a1de8b523d632c37691de8f61"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.992057 10805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 06:11:45.995254 10831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:45.996713 10805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:11:45.996837 10805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "b9a5e83955a344439ef076585e0370c2"
format_stamp: "Formatted at 2026-08-07 06:11:45 on dist-test-slave-h3ft"
server_key: "68462a22e9b91ef763b903c0d0faa27d"
server_key_iv: "44f0496a1de8b523d632c37691de8f61"
server_key_version: "encryptionkey@0"
I20260807 06:11:45.996944 10805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:46.021656 10805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:46.025766 10805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:46.026068 10805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:46.026889 10805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:11:46.028014 10805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:46.028066 10805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:46.028143 10805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:46.028175 10805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:46.039358 10805 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:34857
I20260807 06:11:46.039386 10989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:34857 every 8 connection(s)
I20260807 06:11:46.040517 10805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:11:46.045161 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10805
I20260807 06:11:46.045298 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:11:46.045729 9750 external_mini_cluster.cc:1428] Setting key 426c0008c39334dd499329eafad08857
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:46.057337 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.042789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:53243 (local address 127.9.133.190:38841)
0807 06:11:46.042994 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.043012 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.044107 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 06:11:46.045149 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.045154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.045233 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.045345 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:46.047265 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.048051 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.049334 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.049729 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.049861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.052997 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:46.053000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:46.053002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:46.053031 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:46.054811 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.055480 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.055483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.055485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.055573 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.056007 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.056010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.056011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.056227 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:46.056343 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.056975 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.057135 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:11:46.058460 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.043312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:38841 (local address 127.9.133.130:53243)
0807 06:11:46.043875 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.043965 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.044906 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.045634 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.045642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.046224 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:46.047064 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.047087 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.048191 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.048194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.049206 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.049215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.049909 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.049913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.050119 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.050873 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:46.050892 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:46.052779 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:46.054953 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:46.054960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:46.054982 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:46.055358 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:46.055692 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:46.055695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:46.055697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:46.055885 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:46.056339 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:46.056347 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:46.056686 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.056902 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.057329 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":140,"threads_started":1}
I20260807 06:11:46.060011 10990 heartbeater.cc:347] Connected to a master server at 127.9.133.190:38841
I20260807 06:11:46.060396 10990 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:11:46.061268 10990 heartbeater.cc:511] Master 127.9.133.190:38841 requested a full tablet report, sending...
I20260807 06:11:46.062832 10534 ts_manager.cc:194] Registered new tserver with Master: b9a5e83955a344439ef076585e0370c2 (127.9.133.130:34857)
I20260807 06:11:46.063654 10534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:53243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:11:46.083206 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:38841
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:46.219303 10998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:46.219621 10998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:46.219700 10998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:46.223803 10998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:46.223858 10998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:46.223943 10998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:11:46.229472 10998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:38841
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:46.230844 10998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:46.232159 10998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:11:46.244367 10998 server_base.cc:1034] running on GCE node
W20260807 06:11:46.244352 11004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:46.244349 11006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:46.244590 11003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:46.245057 10998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:46.245895 10998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:46.247098 10998 hybrid_clock.cc:648] HybridClock initialized: now 1786083106247082 us; error 30 us; skew 500 ppm
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:46.250424 10998 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:11:46.251701 10998 webserver.cc:460] Webserver started at http://127.9.133.131:35501/ using document root <none> and password file <none>
I20260807 06:11:46.252414 10998 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:46.252482 10998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:46.252735 10998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:46.255210 10998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "646a4469869848598eb1dadbc4bae2ac"
format_stamp: "Formatted at 2026-08-07 06:11:46 on dist-test-slave-h3ft"
server_key: "c8feda4efe0536df27a45397c409ce67"
server_key_iv: "1b4f1045ba3471a36681d41adb13c8c0"
server_key_version: "encryptionkey@0"
I20260807 06:11:46.255863 10998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "646a4469869848598eb1dadbc4bae2ac"
format_stamp: "Formatted at 2026-08-07 06:11:46 on dist-test-slave-h3ft"
server_key: "c8feda4efe0536df27a45397c409ce67"
server_key_iv: "1b4f1045ba3471a36681d41adb13c8c0"
server_key_version: "encryptionkey@0"
I20260807 06:11:46.260179 10998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:11:46.263228 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:46.264657 10998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 06:11:46.264789 10998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "646a4469869848598eb1dadbc4bae2ac"
format_stamp: "Formatted at 2026-08-07 06:11:46 on dist-test-slave-h3ft"
server_key: "c8feda4efe0536df27a45397c409ce67"
server_key_iv: "1b4f1045ba3471a36681d41adb13c8c0"
server_key_version: "encryptionkey@0"
I20260807 06:11:46.264914 10998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:46.280126 10998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:46.283769 10998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:46.283999 10998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:46.284703 10998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:11:46.285674 10998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:46.285728 10998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:46.285771 10998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:46.285818 10998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:46.296828 10998 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:45325
I20260807 06:11:46.296876 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:45325 every 8 connection(s)
I20260807 06:11:46.297956 10998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:11:46.302438 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10998
I20260807 06:11:46.302598 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:11:46.303013 9750 external_mini_cluster.cc:1428] Setting key e2d4f064d42f1cf50d8e79bdee23e44d
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:46.313969 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.300313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:51187 (local address 127.9.133.190:38841)
0807 06:11:46.300487 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.300492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.301579 (+ 1087us) server_negotiation.cc:407] Connection header received
0807 06:11:46.302984 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.302988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.303102 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.303194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:46.304935 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.305633 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.306858 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.307179 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.307291 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.310190 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:46.310194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:46.310196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:46.310224 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:46.311959 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.312460 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.312462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.312464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.312544 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.312866 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.312869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.312870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.313078 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:46.313153 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.313692 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.313814 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:11:46.315104 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.300787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:38841 (local address 127.9.133.131:51187)
0807 06:11:46.301388 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.301439 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.302660 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.303417 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.303428 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.303990 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:46.304789 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.304808 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.305758 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.305762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.306730 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.306739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.307346 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.307350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.307542 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.308282 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:46.308300 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:46.309985 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:46.312077 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:46.312082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:46.312097 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:46.312374 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:46.312632 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:46.312635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:46.312636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:46.312783 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:46.313175 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:46.313181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:46.313458 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.313634 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.313934 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":134,"threads_started":1}
I20260807 06:11:46.316573 11186 heartbeater.cc:347] Connected to a master server at 127.9.133.190:38841
I20260807 06:11:46.316888 11186 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:46.317672 11186 heartbeater.cc:511] Master 127.9.133.190:38841 requested a full tablet report, sending...
I20260807 06:11:46.318858 10534 ts_manager.cc:194] Registered new tserver with Master: 646a4469869848598eb1dadbc4bae2ac (127.9.133.131:45325)
I20260807 06:11:46.319610 10534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:51187
I20260807 06:11:46.329715 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:11:46.343941 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.335293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.9.133.190:38841)
0807 06:11:46.335487 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.335493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.335631 (+ 138us) server_negotiation.cc:407] Connection header received
0807 06:11:46.335817 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.335821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.335880 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.335981 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:46.336853 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.337484 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.338364 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.338614 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.338724 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.340363 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:46.340366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:46.340370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:46.340402 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:46.341994 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.342452 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.342457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.342459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.342550 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.342831 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.342834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.342836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.343187 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:46.343343 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.343596 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.343734 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:11:46.351575 10534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:11:46.355899 10534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:46.375032 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.369333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:45325 (local address 127.0.0.1:48044)
0807 06:11:46.369738 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.369752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.369897 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.370706 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.370708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.370726 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:46.370817 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.370823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.372128 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.372131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.373324 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.373334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.374374 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.374379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.374499 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.374547 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.374682 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.374824 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":201,"thread_start_us":115,"threads_started":1}
I20260807 06:11:46.375514 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.368012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43889 (local address 127.0.0.1:42322)
0807 06:11:46.369100 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.369160 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.369462 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.370112 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.370118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.370143 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:46.370311 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.370317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.371976 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.371982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.373661 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.373672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.375061 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.375065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.375163 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.375195 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.375274 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.375348 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":166,"thread_start_us":105,"threads_started":1}
I20260807 06:11:46.375618 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.368456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.9.133.130:34857)
0807 06:11:46.368946 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.368959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.369365 (+ 406us) server_negotiation.cc:407] Connection header received
0807 06:11:46.369670 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.369679 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.369940 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.370086 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:46.370590 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.371808 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.373821 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.374661 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.374764 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.374924 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.375309 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.375446 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":196,"threads_started":1}
I20260807 06:11:46.376003 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.368129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34857 (local address 127.0.0.1:36250)
0807 06:11:46.369258 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.369272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.369468 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.370330 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.370333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.370350 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:46.370470 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.370476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.371973 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.371976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.373665 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.373672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.374993 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.375018 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.375133 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.375158 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.375822 (+ 664us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.375901 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":180,"thread_start_us":143,"threads_started":1}
I20260807 06:11:46.376044 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.368196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.9.133.129:43889)
0807 06:11:46.368648 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.368657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.369327 (+ 670us) server_negotiation.cc:407] Connection header received
0807 06:11:46.369701 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.369711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.369942 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.370111 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:46.370442 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.371809 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.373805 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.374680 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.374786 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.374948 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.375762 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.375875 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":73,"threads_started":1}
I20260807 06:11:46.376222 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.369507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.9.133.131:45325)
0807 06:11:46.370231 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.370248 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.370273 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:11:46.370357 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.370361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.370558 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.370728 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:46.370905 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.371967 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.373466 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.374227 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.374342 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.374589 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.374751 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.375902 (+ 1151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":204,"threads_started":1}
I20260807 06:11:46.379683 10702 tablet_service.cc:1467] Processing CreateTablet for tablet dccaf27c0c9b412d8d50c72025917e42 (DEFAULT_TABLE table=test-table [id=716a9d3345a04336926703fd42c2193a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:46.381134 10702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dccaf27c0c9b412d8d50c72025917e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:46.380827 10895 tablet_service.cc:1467] Processing CreateTablet for tablet dccaf27c0c9b412d8d50c72025917e42 (DEFAULT_TABLE table=test-table [id=716a9d3345a04336926703fd42c2193a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:46.382050 10895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dccaf27c0c9b412d8d50c72025917e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:46.382004 11083 tablet_service.cc:1467] Processing CreateTablet for tablet dccaf27c0c9b412d8d50c72025917e42 (DEFAULT_TABLE table=test-table [id=716a9d3345a04336926703fd42c2193a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:46.383160 11083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dccaf27c0c9b412d8d50c72025917e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:46.389008 11208 tablet_bootstrap.cc:492] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: Bootstrap starting.
I20260807 06:11:46.391137 11209 tablet_bootstrap.cc:492] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: Bootstrap starting.
I20260807 06:11:46.391440 11210 tablet_bootstrap.cc:492] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: Bootstrap starting.
I20260807 06:11:46.393049 11208 tablet_bootstrap.cc:654] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:46.393697 11210 tablet_bootstrap.cc:654] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:46.393976 11208 log.cc:824] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:46.394454 11209 tablet_bootstrap.cc:654] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:46.395253 11210 log.cc:824] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:46.395294 11209 log.cc:824] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:46.396364 11208 tablet_bootstrap.cc:492] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: No bootstrap required, opened a new log
I20260807 06:11:46.396595 11208 ts_tablet_manager.cc:1397] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 06:11:46.397436 11209 tablet_bootstrap.cc:492] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: No bootstrap required, opened a new log
I20260807 06:11:46.397645 11209 ts_tablet_manager.cc:1397] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:11:46.397825 11210 tablet_bootstrap.cc:492] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: No bootstrap required, opened a new log
I20260807 06:11:46.398020 11210 ts_tablet_manager.cc:1397] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 06:11:46.401163 11208 raft_consensus.cc:348] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.401506 11208 raft_consensus.cc:374] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:46.401582 11208 raft_consensus.cc:729] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f40d4f7622147fe85ac6fd99c35d18f, State: Initialized, Role: FOLLOWER
I20260807 06:11:46.402102 11208 consensus_queue.cc:260] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.402128 11210 raft_consensus.cc:348] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.402158 11209 raft_consensus.cc:348] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.402449 11209 raft_consensus.cc:374] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:46.402449 11210 raft_consensus.cc:374] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:46.402518 11210 raft_consensus.cc:729] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646a4469869848598eb1dadbc4bae2ac, State: Initialized, Role: FOLLOWER
I20260807 06:11:46.402529 11209 raft_consensus.cc:729] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a5e83955a344439ef076585e0370c2, State: Initialized, Role: FOLLOWER
I20260807 06:11:46.403023 11209 consensus_queue.cc:260] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.403017 11210 consensus_queue.cc:260] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.403402 11208 ts_tablet_manager.cc:1428] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 06:11:46.403904 11186 heartbeater.cc:503] Master 127.9.133.190:38841 was elected leader, sending a full tablet report...
I20260807 06:11:46.404045 11209 ts_tablet_manager.cc:1428] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:11:46.405160 10800 heartbeater.cc:503] Master 127.9.133.190:38841 was elected leader, sending a full tablet report...
I20260807 06:11:46.405261 10990 heartbeater.cc:503] Master 127.9.133.190:38841 was elected leader, sending a full tablet report...
I20260807 06:11:46.406462 11210 ts_tablet_manager.cc:1428] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:11:46.448349 11217 raft_consensus.cc:482] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:46.448643 11217 raft_consensus.cc:504] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.450167 11217 leader_election.cc:288] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f40d4f7622147fe85ac6fd99c35d18f (127.9.133.129:43889), 646a4469869848598eb1dadbc4bae2ac (127.9.133.131:45325)
I20260807 06:11:46.454747 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.450553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:40813 (local address 127.9.133.131:45325)
0807 06:11:46.450733 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.450738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.450759 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:11:46.450831 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.450839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.450891 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.450974 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:46.451391 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.452175 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.453471 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.454109 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.454227 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.454281 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.454550 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.454620 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 06:11:46.454769 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.450434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:45325 (local address 127.9.133.130:40813)
0807 06:11:46.450570 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.450589 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.450715 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.451041 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.451047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.451076 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:46.451241 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.451249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.452298 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.452302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.453331 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.453341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.454262 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.454267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.454392 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.454423 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.454523 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.454605 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 06:11:46.455626 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.450623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:33153 (local address 127.9.133.129:43889)
0807 06:11:46.450786 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.450792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.451455 (+ 663us) server_negotiation.cc:407] Connection header received
0807 06:11:46.451600 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.451602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.451655 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.451719 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:46.452050 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.452756 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.454152 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.454863 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.455031 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.455099 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.455416 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.455505 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:11:46.455679 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.450448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43889 (local address 127.9.133.130:33153)
0807 06:11:46.451339 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:46.451355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:46.451499 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:46.451754 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:46.451757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:46.451777 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:46.451936 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.451942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.452875 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.452877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.454004 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:46.454015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.455163 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.455167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:46.455274 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.455295 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 06:11:46.455388 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:11:46.455463 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":330,"thread_start_us":265,"threads_started":1}
I20260807 06:11:46.455868 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccaf27c0c9b412d8d50c72025917e42" candidate_uuid: "b9a5e83955a344439ef076585e0370c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646a4469869848598eb1dadbc4bae2ac" is_pre_election: true
I20260807 06:11:46.456427 11111 raft_consensus.cc:2454] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a5e83955a344439ef076585e0370c2 in term 0.
I20260807 06:11:46.456460 10727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccaf27c0c9b412d8d50c72025917e42" candidate_uuid: "b9a5e83955a344439ef076585e0370c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" is_pre_election: true
I20260807 06:11:46.456777 10727 raft_consensus.cc:2454] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a5e83955a344439ef076585e0370c2 in term 0.
I20260807 06:11:46.457088 10835 leader_election.cc:302] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 646a4469869848598eb1dadbc4bae2ac, b9a5e83955a344439ef076585e0370c2; no voters:
I20260807 06:11:46.457425 11217 raft_consensus.cc:2781] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:46.457526 11217 raft_consensus.cc:482] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:46.457604 11217 raft_consensus.cc:3037] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:46.459033 11217 raft_consensus.cc:504] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.459472 11217 leader_election.cc:288] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [CANDIDATE]: Term 1 election: Requested vote from peers 8f40d4f7622147fe85ac6fd99c35d18f (127.9.133.129:43889), 646a4469869848598eb1dadbc4bae2ac (127.9.133.131:45325)
I20260807 06:11:46.459879 11111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccaf27c0c9b412d8d50c72025917e42" candidate_uuid: "b9a5e83955a344439ef076585e0370c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646a4469869848598eb1dadbc4bae2ac"
I20260807 06:11:46.459890 10727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccaf27c0c9b412d8d50c72025917e42" candidate_uuid: "b9a5e83955a344439ef076585e0370c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f40d4f7622147fe85ac6fd99c35d18f"
I20260807 06:11:46.460050 11111 raft_consensus.cc:3037] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:46.460062 10727 raft_consensus.cc:3037] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:46.461385 10727 raft_consensus.cc:2454] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a5e83955a344439ef076585e0370c2 in term 1.
I20260807 06:11:46.461526 11111 raft_consensus.cc:2454] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a5e83955a344439ef076585e0370c2 in term 1.
I20260807 06:11:46.461793 10837 leader_election.cc:302] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f40d4f7622147fe85ac6fd99c35d18f, b9a5e83955a344439ef076585e0370c2; no voters:
I20260807 06:11:46.461966 11217 raft_consensus.cc:2781] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:46.462239 11217 raft_consensus.cc:686] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [term 1 LEADER]: Becoming Leader. State: Replica: b9a5e83955a344439ef076585e0370c2, State: Running, Role: LEADER
I20260807 06:11:46.462724 11217 consensus_queue.cc:237] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } }
I20260807 06:11:46.466893 10529 catalog_manager.cc:5495] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9a5e83955a344439ef076585e0370c2 (127.9.133.130). New cstate: current_term: 1 leader_uuid: "b9a5e83955a344439ef076585e0370c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f40d4f7622147fe85ac6fd99c35d18f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9a5e83955a344439ef076585e0370c2" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "646a4469869848598eb1dadbc4bae2ac" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45325 } health_report { overall_health: UNKNOWN } } }
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:11:46.510291 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.493350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.9.133.129:43889)
0807 06:11:46.493567 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.493573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.493742 (+ 169us) server_negotiation.cc:407] Connection header received
0807 06:11:46.493930 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.493934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.494002 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.494098 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:46.494950 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.495705 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.496721 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.496966 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.497077 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.506028 (+ 8951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:46.506049 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:46.506070 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:46.506110 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:46.508279 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.508783 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.508790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.508791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.508889 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.509179 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.509184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.509186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.509520 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:46.509779 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.509943 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.510094 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 06:11:46.526675 10801 tablet.cc:2369] T dccaf27c0c9b412d8d50c72025917e42 P 8f40d4f7622147fe85ac6fd99c35d18f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083106, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083106, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
W20260807 06:11:46.541865 10991 tablet.cc:2369] T dccaf27c0c9b412d8d50c72025917e42 P b9a5e83955a344439ef076585e0370c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:11:46.545423 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.535621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.9.133.129:43889)
0807 06:11:46.535803 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.535808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.535975 (+ 167us) server_negotiation.cc:407] Connection header received
0807 06:11:46.536126 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.536129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.536184 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.536263 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:46.537076 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.537656 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.538486 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.538714 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.538796 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.541625 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:46.541630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:46.541632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:46.541660 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:46.543415 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.543903 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.543908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.543911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.544006 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.544317 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.544322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.544324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.544539 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:46.544697 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.545050 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.545202 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 06:11:46.549166 11187 tablet.cc:2369] T dccaf27c0c9b412d8d50c72025917e42 P 646a4469869848598eb1dadbc4bae2ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[10468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:11:46.576836 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:46.567472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.9.133.129:43889)
0807 06:11:46.567652 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:46.567658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:46.567816 (+ 158us) server_negotiation.cc:407] Connection header received
0807 06:11:46.567965 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:46.567968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:46.568023 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:46.568101 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:46.568901 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.569451 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.570506 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:46.570730 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:46.570811 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:46.573584 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:46.573587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:46.573589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:46.573618 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:46.575277 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.575699 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.575702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.575703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.575781 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:46.576018 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:46.576020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:46.576021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:46.576198 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:46.576301 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:46.576505 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 06:11:46.576615 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 06:11:46.578969 10673 tablet_service.cc:2908] Rejecting scan request for tablet dccaf27c0c9b412d8d50c72025917e42: Uninitialized: clean time has not yet been initialized
I20260807 06:11:46.581787 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10613
I20260807 06:11:46.599632 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10805
I20260807 06:11:46.609676 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10998
I20260807 06:11:46.623073 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10490
2026-08-07T06:11:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2817 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[11245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[11245](info): set up 2 sockets
Aug 07 06:11:46 dist-test-slave-h3ft krb5kdc[11245](info): commencing operation
krb5kdc: starting...
W20260807 06:11:47.915084 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:11:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:11:48.074854 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42915
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:37335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:48.208702 11265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:48.209101 11265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:48.209180 11265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:48.213376 11265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:11:48.213486 11265 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:11:48.213529 11265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:48.213573 11265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:48.213614 11265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:11:48.213662 11265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:11:48.219672 11265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:37335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42915
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:48.221529 11265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:48.223320 11265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:48.233521 11271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:48.233508 11272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:48.233532 11275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:48.233912 11265 server_base.cc:1034] running on GCE node
I20260807 06:11:48.234699 11265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:48.236124 11265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:48.237303 11265 hybrid_clock.cc:648] HybridClock initialized: now 1786083108237283 us; error 33 us; skew 500 ppm
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:48.246137 11265 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:11:48.247454 11265 webserver.cc:460] Webserver started at http://127.9.133.190:33913/ using document root <none> and password file <none>
I20260807 06:11:48.248184 11265 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:48.248275 11265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:48.248548 11265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:48.251197 11265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "9d3b2b282dae477b915f8be542223429"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "2c549a74f26a0e48f80bed6ced280c38"
server_key_iv: "590aae877305a870cc512175ac807c23"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.251883 11265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "9d3b2b282dae477b915f8be542223429"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "2c549a74f26a0e48f80bed6ced280c38"
server_key_iv: "590aae877305a870cc512175ac807c23"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.256565 11265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:11:48.259445 11286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:48.260910 11265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:11:48.261052 11265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "9d3b2b282dae477b915f8be542223429"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "2c549a74f26a0e48f80bed6ced280c38"
server_key_iv: "590aae877305a870cc512175ac807c23"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.261165 11265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:48.283718 11265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:48.287387 11265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:48.287590 11265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:48.298942 11265 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:42915
I20260807 06:11:48.298970 11380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:42915 every 8 connection(s)
I20260807 06:11:48.300163 11265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:11:48.303740 11381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:48.304514 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 11265
I20260807 06:11:48.304661 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:11:48.305027 9750 external_mini_cluster.cc:1428] Setting key 067eb05ed8402462d221c746c7022612
I20260807 06:11:48.312304 11381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: Bootstrap starting.
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:48.315860 11381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:48.317677 11381 log.cc:824] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:48.320616 11381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: No bootstrap required, opened a new log
I20260807 06:11:48.321826 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.306705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.9.133.190:42915)
0807 06:11:48.307199 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:48.307218 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:48.307262 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:11:48.308010 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:48.308033 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:48.308414 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:48.308748 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:48.309738 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.310828 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.312068 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.312634 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.312763 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.316001 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:48.316008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:48.316031 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:48.316067 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:48.318315 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.318849 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.318857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.318864 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.319062 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.319328 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.319330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.319332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.319913 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:48.320148 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:48.320355 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 06:11:48.320705 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":148,"threads_started":1}
I20260807 06:11:48.324759 11381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b2b282dae477b915f8be542223429" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42915 } }
I20260807 06:11:48.325013 11381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:48.325050 11381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3b2b282dae477b915f8be542223429, State: Initialized, Role: FOLLOWER
I20260807 06:11:48.325470 11381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b2b282dae477b915f8be542223429" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42915 } }
I20260807 06:11:48.325603 11381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:11:48.325650 11381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:11:48.325729 11381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:48.326753 11381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b2b282dae477b915f8be542223429" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42915 } }
I20260807 06:11:48.327155 11381 leader_election.cc:302] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d3b2b282dae477b915f8be542223429; no voters:
I20260807 06:11:48.327471 11381 leader_election.cc:288] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:11:48.327653 11387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:48.327931 11387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3b2b282dae477b915f8be542223429, State: Running, Role: LEADER
I20260807 06:11:48.328262 11387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b2b282dae477b915f8be542223429" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42915 } }
I20260807 06:11:48.328935 11381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:11:48.330427 11389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d3b2b282dae477b915f8be542223429. Latest consensus state: current_term: 1 leader_uuid: "9d3b2b282dae477b915f8be542223429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b2b282dae477b915f8be542223429" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42915 } } }
I20260807 06:11:48.330425 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d3b2b282dae477b915f8be542223429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b2b282dae477b915f8be542223429" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42915 } } }
I20260807 06:11:48.330607 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:48.330606 11389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429 [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:48.331081 11403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:11:48.334887 11403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:11:48.341598 11403 catalog_manager.cc:1347] Generated new cluster ID: 30d2336742264dec909d0b8cbffaaa8b
I20260807 06:11:48.341694 11403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:11:48.351992 11403 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:11:48.353600 11403 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:11:48.364722 11403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: Generated new TSK 0
I20260807 06:11:48.365803 11403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:11:48.406920 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:42915
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:37335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:48.548758 11426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:48.549117 11426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:48.549170 11426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:48.554317 11426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:48.554397 11426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:48.554469 11426 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:11:48.554545 11426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:11:48.560703 11426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:37335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:42915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:48.562176 11426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:48.563551 11426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:48.575461 11435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:48.575443 11433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:48.575450 11432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:48.575850 11426 server_base.cc:1034] running on GCE node
I20260807 06:11:48.576309 11426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:48.577097 11426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:48.578298 11426 hybrid_clock.cc:648] HybridClock initialized: now 1786083108578273 us; error 38 us; skew 500 ppm
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:48.581534 11426 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:11:48.582564 11426 webserver.cc:460] Webserver started at http://127.9.133.129:41121/ using document root <none> and password file <none>
I20260807 06:11:48.583314 11426 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:48.583413 11426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:48.583694 11426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:48.586119 11426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "c11c4c6e2d34422697f5245561a2839f"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "adb7187a37f0ecf73b335dd0120a440e"
server_key_iv: "982bd4f050e44aa134551d0a628807fa"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.586880 11426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "c11c4c6e2d34422697f5245561a2839f"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "adb7187a37f0ecf73b335dd0120a440e"
server_key_iv: "982bd4f050e44aa134551d0a628807fa"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.591339 11426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 06:11:48.594370 11443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:48.595885 11426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:11:48.596053 11426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "c11c4c6e2d34422697f5245561a2839f"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "adb7187a37f0ecf73b335dd0120a440e"
server_key_iv: "982bd4f050e44aa134551d0a628807fa"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.596191 11426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:48.613749 11426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:48.617496 11426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:48.617731 11426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:48.619596 11426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:48.619673 11426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:48.619752 11426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:48.619803 11426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:48.633327 11426 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:45579
I20260807 06:11:48.633397 11607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:45579 every 8 connection(s)
I20260807 06:11:48.634536 11426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:11:48.636420 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.620572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:58159 (local address 127.9.133.190:42915)
0807 06:11:48.620724 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:48.620728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:48.621917 (+ 1189us) server_negotiation.cc:407] Connection header received
0807 06:11:48.623164 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:48.623171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:48.623257 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:48.623332 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:48.624815 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.625586 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.627074 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.627354 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.627493 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.631931 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:48.631934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:48.631942 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:48.631975 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:48.633849 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.634466 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.634470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.634472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.634570 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.634949 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.634952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.634954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.635297 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:48.635404 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:48.636104 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 06:11:48.636264 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:11:48.637069 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 11426
I20260807 06:11:48.637212 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:11:48.637442 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.621154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.9.133.129:58159)
0807 06:11:48.621745 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:48.621794 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:48.622810 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:48.623743 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:48.623750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:48.624116 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:48.624645 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.624659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.625736 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.625740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.626908 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.626921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.627537 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.627541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.627786 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.628694 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:48.628711 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:48.631712 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:48.633989 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.633996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.634017 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.634334 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.634671 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.634675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.634677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.634840 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.635463 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:48.635470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:48.635801 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 06:11:48.635986 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 06:11:48.636306 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":182,"threads_started":1}
I20260807 06:11:48.637578 9750 external_mini_cluster.cc:1428] Setting key 879d32501ddac6dd111977fa38206e24
I20260807 06:11:48.639144 11608 heartbeater.cc:347] Connected to a master server at 127.9.133.190:42915
I20260807 06:11:48.639322 11608 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:48.640038 11608 heartbeater.cc:511] Master 127.9.133.190:42915 requested a full tablet report, sending...
I20260807 06:11:48.643078 11306 ts_manager.cc:194] Registered new tserver with Master: c11c4c6e2d34422697f5245561a2839f (127.9.133.129:45579)
I20260807 06:11:48.644536 11306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:58159
I20260807 06:11:48.649870 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.641752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:43063 (local address 127.9.133.190:42915)
0807 06:11:48.641911 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:48.641916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:48.642185 (+ 269us) server_negotiation.cc:407] Connection header received
0807 06:11:48.642311 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:48.642314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:48.642364 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:48.642453 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:48.643103 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.643881 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.644794 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.645041 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.645157 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.646425 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:48.646428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:48.646431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:48.646464 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:48.648137 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.648603 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.648606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.648608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.648699 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.648985 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.648990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.648992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.649185 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:48.649297 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:48.649569 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 06:11:48.649693 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:11:48.649876 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.641668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.9.133.129:43063)
0807 06:11:48.642068 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:48.642087 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:48.642214 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:48.642489 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:48.642492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:48.642759 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:48.642924 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.642932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.644033 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.644036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.644662 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.644674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.645163 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.645167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.645270 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.645879 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:48.645892 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:48.646273 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:48.648269 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.648273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.648275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.648492 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.648792 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.648796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.648798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.648892 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.649284 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:48.649288 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:48.649393 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 06:11:48.649505 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:11:48.649666 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:11:48.674729 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:42915
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:37335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:48.811185 11618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:48.811600 11618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:48.811658 11618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:48.816289 11618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:48.816377 11618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:48.816424 11618 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:11:48.816491 11618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:11:48.822702 11618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:37335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:42915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:48.824211 11618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:48.825590 11618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:48.837810 11626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:48.837808 11624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:48.837838 11623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:48.838250 11618 server_base.cc:1034] running on GCE node
I20260807 06:11:48.838701 11618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:48.839612 11618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:48.840780 11618 hybrid_clock.cc:648] HybridClock initialized: now 1786083108840755 us; error 36 us; skew 500 ppm
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:48.845037 11618 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:11:48.846771 11618 webserver.cc:460] Webserver started at http://127.9.133.130:39003/ using document root <none> and password file <none>
I20260807 06:11:48.847807 11618 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:48.847924 11618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:48.848233 11618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:48.851380 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "0dff49a14bb140809e083a035dc18f12"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "600a34349d4bfbcb1aff49e7f1c78a77"
server_key_iv: "b67e6027bae19910f48438fa8699116c"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.852102 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "0dff49a14bb140809e083a035dc18f12"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "600a34349d4bfbcb1aff49e7f1c78a77"
server_key_iv: "b67e6027bae19910f48438fa8699116c"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.856887 11618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:11:48.860028 11638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:48.861563 11618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:11:48.861720 11618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "0dff49a14bb140809e083a035dc18f12"
format_stamp: "Formatted at 2026-08-07 06:11:48 on dist-test-slave-h3ft"
server_key: "600a34349d4bfbcb1aff49e7f1c78a77"
server_key_iv: "b67e6027bae19910f48438fa8699116c"
server_key_version: "encryptionkey@0"
I20260807 06:11:48.861855 11618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:48.885716 11618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:48.890054 11618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:48.890319 11618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:48.892336 11618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:48.892395 11618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:48.892443 11618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:48.892504 11618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:11:48 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:48.906100 11618 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:39667
I20260807 06:11:48.906174 11797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:39667 every 8 connection(s)
I20260807 06:11:48.907341 11618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:11:48.909013 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.893414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:38531 (local address 127.9.133.190:42915)
0807 06:11:48.893585 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:48.893590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:48.894845 (+ 1255us) server_negotiation.cc:407] Connection header received
0807 06:11:48.896186 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:48.896191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:48.896268 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:48.896377 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:48.898493 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.899239 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.900665 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.900917 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.901063 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.904593 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:48.904597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:48.904599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:48.904629 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:48.906457 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.907073 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.907077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.907079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.907180 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.907604 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.907608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.907610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.907945 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:48.908054 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:48.908718 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 06:11:48.908847 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:11:48.910022 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.894063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.9.133.130:38531)
0807 06:11:48.894659 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:48.894709 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:48.895891 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:48.896696 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:48.896709 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:48.897329 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:48.898257 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.898290 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.899379 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.899383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.900531 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.900542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.901134 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.901142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.901452 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.902310 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:48.902333 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:48.904374 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:48.906601 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.906607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.906623 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.906917 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.907283 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.907286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.907288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.907471 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.908115 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:48.908126 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:48.908469 (+ 343us) client_negotiation.cc:769] Sending connection context
0807 06:11:48.908645 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 06:11:48.908951 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":165,"threads_started":1}
I20260807 06:11:48.911554 11799 heartbeater.cc:347] Connected to a master server at 127.9.133.190:42915
I20260807 06:11:48.911728 11799 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:48.912421 11799 heartbeater.cc:511] Master 127.9.133.190:42915 requested a full tablet report, sending...
I20260807 06:11:48.913942 11299 ts_manager.cc:194] Registered new tserver with Master: 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667)
I20260807 06:11:48.914371 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 11618
I20260807 06:11:48.914521 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:11:48.914876 9750 external_mini_cluster.cc:1428] Setting key 4a201e1eb761d1e130d563cddbeda05d
I20260807 06:11:48.914945 11299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:38531
I20260807 06:11:48.923435 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.914557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:38883 (local address 127.9.133.190:42915)
0807 06:11:48.914711 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:48.914716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:48.915118 (+ 402us) server_negotiation.cc:407] Connection header received
0807 06:11:48.915434 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:48.915438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:48.915502 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:48.915585 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:48.916241 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.916912 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.917745 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.918138 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.918262 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.919693 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:48.919696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:48.919699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:48.919723 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:48.921446 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.921982 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.921985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.921987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.922069 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:48.922373 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:48.922376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:48.922377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:48.922640 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:48.922723 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:48.923119 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 06:11:48.923281 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:11:48.924680 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:48.914617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.9.133.130:38883)
0807 06:11:48.914992 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:48.915041 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:48.915201 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:48.915627 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:48.915632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:48.915911 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:48.916093 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.916102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.917068 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.917071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.917628 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:48.917635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:48.918259 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:48.918261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:48.918334 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:48.918968 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:48.918980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:48.919546 (+ 566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:48.921569 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.921575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.921580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.921881 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.922161 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:48.922166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:48.922171 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:48.922284 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:48.922740 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:48.922746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:48.922874 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 06:11:48.923038 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 06:11:48.924437 (+ 1399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":1,"thread_start_us":174,"threads_started":1}
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:11:48.954577 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:42915
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:37335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:49.095350 11809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:49.095783 11809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:49.095865 11809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:49.100692 11809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:49.100822 11809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:49.100904 11809 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:11:49.100994 11809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:11:49.107414 11809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:37335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:42915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:49.109010 11809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:49.110432 11809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:49.122773 11820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:49.122852 11809 server_base.cc:1034] running on GCE node
W20260807 06:11:49.122800 11817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:49.122800 11818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:49.123600 11809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:49.124469 11809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:49.125680 11809 hybrid_clock.cc:648] HybridClock initialized: now 1786083109125661 us; error 28 us; skew 500 ppm
Aug 07 06:11:49 dist-test-slave-h3ft krb5kdc[11245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:49.129216 11809 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:11:49.130491 11809 webserver.cc:460] Webserver started at http://127.9.133.131:37481/ using document root <none> and password file <none>
I20260807 06:11:49.131332 11809 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:49.131443 11809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:49.131729 11809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:49.134348 11809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "2fdb001829b742388fde98c74a3641f9"
format_stamp: "Formatted at 2026-08-07 06:11:49 on dist-test-slave-h3ft"
server_key: "1e06ea2521e0e904cc6973165d3aadc6"
server_key_iv: "40e278ff9bc8ed4c2fd84ebd59cf9db2"
server_key_version: "encryptionkey@0"
I20260807 06:11:49.135133 11809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "2fdb001829b742388fde98c74a3641f9"
format_stamp: "Formatted at 2026-08-07 06:11:49 on dist-test-slave-h3ft"
server_key: "1e06ea2521e0e904cc6973165d3aadc6"
server_key_iv: "40e278ff9bc8ed4c2fd84ebd59cf9db2"
server_key_version: "encryptionkey@0"
I20260807 06:11:49.140057 11809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 06:11:49.143325 11829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:49.144994 11809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:11:49.145162 11809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "2fdb001829b742388fde98c74a3641f9"
format_stamp: "Formatted at 2026-08-07 06:11:49 on dist-test-slave-h3ft"
server_key: "1e06ea2521e0e904cc6973165d3aadc6"
server_key_iv: "40e278ff9bc8ed4c2fd84ebd59cf9db2"
server_key_version: "encryptionkey@0"
I20260807 06:11:49.145306 11809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:49.160323 11809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:49.164328 11809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:49.164572 11809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:49.166496 11809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:49.166574 11809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:49.166692 11809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:49.166728 11809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:11:49 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:49.179622 11809 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:44731
I20260807 06:11:49.179699 11985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:44731 every 8 connection(s)
I20260807 06:11:49.180969 11809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:11:49.183871 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.167598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:41637 (local address 127.9.133.190:42915)
0807 06:11:49.167785 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.167790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.168986 (+ 1196us) server_negotiation.cc:407] Connection header received
0807 06:11:49.170565 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.170572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.170653 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.170748 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:49.172285 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.173291 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.174922 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.175323 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.175469 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.179878 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:49.179882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:49.179886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:49.179920 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:49.181745 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:49.182298 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:49.182301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:49.182302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:49.182398 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:49.182669 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:49.182671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:49.182673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:49.182880 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:49.182965 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.183587 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.183719 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 06:11:49.184556 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 11809
I20260807 06:11:49.184705 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:11:49.185077 9750 external_mini_cluster.cc:1428] Setting key 342cc00f0bcac32ee643593c771087ec
I20260807 06:11:49.185127 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.168183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.9.133.131:41637)
0807 06:11:49.168756 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.168827 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.170234 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.170963 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.170976 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.171523 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:49.172082 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.172097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.173463 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.173468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.174749 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.174779 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.175536 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.175543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.175815 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.177443 (+ 1628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:49.177475 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:49.179619 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:49.181902 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:49.181907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:49.181923 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:49.182202 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:49.182466 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:49.182467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:49.182469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:49.182592 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:49.182975 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:49.182980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:49.183338 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.183516 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.183808 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":126,"threads_started":1}
I20260807 06:11:49.186600 11986 heartbeater.cc:347] Connected to a master server at 127.9.133.190:42915
I20260807 06:11:49.186765 11986 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:49.187511 11986 heartbeater.cc:511] Master 127.9.133.190:42915 requested a full tablet report, sending...
I20260807 06:11:49.188679 11306 ts_manager.cc:194] Registered new tserver with Master: 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:49.189560 11306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:41637
I20260807 06:11:49.196632 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.189241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:56139 (local address 127.9.133.190:42915)
0807 06:11:49.189380 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.189385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.189560 (+ 175us) server_negotiation.cc:407] Connection header received
0807 06:11:49.189728 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.189731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.189783 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.189872 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:49.190357 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.190902 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.191783 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.192018 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.192129 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.193375 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:49.193379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:49.193382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:49.193408 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:49.194824 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:49.195360 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:49.195365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:49.195367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:49.195457 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:49.195742 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:49.195747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:49.195749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:49.195950 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:49.196063 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.196332 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.196463 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:11:49.196645 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.189108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.9.133.131:56139)
0807 06:11:49.189462 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.189478 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.189593 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.189889 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.189892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.190132 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:49.190243 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.190248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.191081 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.191084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.191654 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.191662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.192134 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.192137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.192234 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.192855 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:49.192867 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:49.193234 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:49.194935 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:49.194940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:49.194943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:49.195258 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:49.195555 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:49.195559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:49.195561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:49.195650 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:49.196049 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:49.196052 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:49.196160 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.196271 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.196436 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
I20260807 06:11:49.199936 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:11:49 dist-test-slave-h3ft krb5kdc[11245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:11:49 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:49.233094 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.222819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.9.133.190:42915)
0807 06:11:49.223071 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.223090 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.223113 (+ 23us) server_negotiation.cc:407] Connection header received
0807 06:11:49.223182 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.223185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.223246 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.223353 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:49.224418 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.225301 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.226706 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.227098 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.227216 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.229844 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:49.229847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:49.229850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:49.229878 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:49.231479 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:49.231895 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:49.231898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:49.231900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:49.232015 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:49.232262 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:49.232264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:49.232266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:49.232462 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:49.232570 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.232801 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.232911 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 06:11:49.237037 11306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:11:49.240962 11306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:49.258683 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.252022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.9.133.129:45579)
0807 06:11:49.252476 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.252507 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.252567 (+ 60us) server_negotiation.cc:407] Connection header received
0807 06:11:49.252820 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.252827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.253222 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.253385 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.255122 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.256166 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.257314 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.257911 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.258045 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.258228 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.258367 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.258498 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":119,"threads_started":1}
I20260807 06:11:49.259043 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.251532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:45579 (local address 127.0.0.1:46046)
0807 06:11:49.252475 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.252489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.252618 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.253366 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.253368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.253382 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.253582 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.253588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.256291 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.256295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.257183 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.257192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.258043 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.258047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.258155 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.258194 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.258335 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.258741 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":126,"thread_start_us":132,"threads_started":1}
I20260807 06:11:49.259395 12018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.252781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.9.133.131:44731)
0807 06:11:49.253345 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.253359 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.253383 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:11:49.253466 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.253471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.253666 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.253840 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.253979 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.255232 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.257338 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.258167 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.258274 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.258442 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.259119 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.259227 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":139,"threads_started":1}
I20260807 06:11:49.259052 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.252164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:44731 (local address 127.0.0.1:34360)
0807 06:11:49.253105 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.253118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.253203 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.253811 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.253814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.253830 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.253902 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.253906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.255365 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.255368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.257186 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.257198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.258513 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.258517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.258674 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.258700 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.258772 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.258848 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":209,"thread_start_us":115,"threads_started":1}
I20260807 06:11:49.259809 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.251341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:39667 (local address 127.0.0.1:55668)
0807 06:11:49.252112 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.252168 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.252324 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.253267 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.253275 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.253306 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.253580 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.253589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.255477 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.255481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.257186 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.257198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.258513 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.258517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.259472 (+ 955us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.259503 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.259580 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.259664 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":894,"thread_start_us":107,"threads_started":1}
I20260807 06:11:49.260246 12014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.251552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.9.133.130:39667)
0807 06:11:49.252284 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.252299 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.252332 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:11:49.252604 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.252612 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.252871 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.253048 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.253713 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.255179 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.257338 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.258167 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.258275 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.258442 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.259991 (+ 1549us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.260092 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":121,"threads_started":1}
I20260807 06:11:49.262499 11896 tablet_service.cc:1467] Processing CreateTablet for tablet 87eb4c2fee184978b7ee060db534a6b7 (DEFAULT_TABLE table=test-table [id=c2fdbf2004a643548f0c83aca4ec8882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:49.262494 11512 tablet_service.cc:1467] Processing CreateTablet for tablet 87eb4c2fee184978b7ee060db534a6b7 (DEFAULT_TABLE table=test-table [id=c2fdbf2004a643548f0c83aca4ec8882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:49.263816 11512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87eb4c2fee184978b7ee060db534a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:49.263828 11896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87eb4c2fee184978b7ee060db534a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:49.265704 11703 tablet_service.cc:1467] Processing CreateTablet for tablet 87eb4c2fee184978b7ee060db534a6b7 (DEFAULT_TABLE table=test-table [id=c2fdbf2004a643548f0c83aca4ec8882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:49.266762 11703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87eb4c2fee184978b7ee060db534a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:49.271257 12019 tablet_bootstrap.cc:492] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Bootstrap starting.
I20260807 06:11:49.274066 12019 tablet_bootstrap.cc:654] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:49.274066 12021 tablet_bootstrap.cc:492] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Bootstrap starting.
I20260807 06:11:49.274955 12019 log.cc:824] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:49.275504 12022 tablet_bootstrap.cc:492] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Bootstrap starting.
I20260807 06:11:49.276549 12021 tablet_bootstrap.cc:654] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:49.277467 12021 log.cc:824] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:49.277593 12022 tablet_bootstrap.cc:654] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:49.277767 12019 tablet_bootstrap.cc:492] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: No bootstrap required, opened a new log
I20260807 06:11:49.278017 12019 ts_tablet_manager.cc:1397] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 06:11:49.278501 12022 log.cc:824] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:49.279901 12021 tablet_bootstrap.cc:492] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: No bootstrap required, opened a new log
I20260807 06:11:49.280117 12021 ts_tablet_manager.cc:1397] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 06:11:49.280320 12022 tablet_bootstrap.cc:492] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: No bootstrap required, opened a new log
I20260807 06:11:49.280519 12022 ts_tablet_manager.cc:1397] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 06:11:49.283802 12022 raft_consensus.cc:348] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.283810 12019 raft_consensus.cc:348] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.284075 12022 raft_consensus.cc:374] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:49.284080 12019 raft_consensus.cc:374] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:49.284147 12022 raft_consensus.cc:729] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dff49a14bb140809e083a035dc18f12, State: Initialized, Role: FOLLOWER
I20260807 06:11:49.284144 12019 raft_consensus.cc:729] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c11c4c6e2d34422697f5245561a2839f, State: Initialized, Role: FOLLOWER
I20260807 06:11:49.284176 12021 raft_consensus.cc:348] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.284415 12021 raft_consensus.cc:374] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:49.284462 12021 raft_consensus.cc:729] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdb001829b742388fde98c74a3641f9, State: Initialized, Role: FOLLOWER
I20260807 06:11:49.284662 12019 consensus_queue.cc:260] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.284672 12022 consensus_queue.cc:260] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.284825 12021 consensus_queue.cc:260] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.285460 11608 heartbeater.cc:503] Master 127.9.133.190:42915 was elected leader, sending a full tablet report...
I20260807 06:11:49.285776 12022 ts_tablet_manager.cc:1428] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 06:11:49.285777 12021 ts_tablet_manager.cc:1428] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 06:11:49.285777 12019 ts_tablet_manager.cc:1428] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 06:11:49.286109 11986 heartbeater.cc:503] Master 127.9.133.190:42915 was elected leader, sending a full tablet report...
I20260807 06:11:49.286113 11799 heartbeater.cc:503] Master 127.9.133.190:42915 was elected leader, sending a full tablet report...
W20260807 06:11:49.386399 11609 tablet.cc:2369] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:11:49.408942 11800 tablet.cc:2369] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:11:49.432220 11988 tablet.cc:2369] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:11:49.637120 12027 raft_consensus.cc:482] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:49.637378 12027 raft_consensus.cc:504] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.638967 12027 leader_election.cc:288] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667), 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:49.644497 12014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.639488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:56643 (local address 127.9.133.130:39667)
0807 06:11:49.639671 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.639676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.639814 (+ 138us) server_negotiation.cc:407] Connection header received
0807 06:11:49.640061 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.640068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.640141 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.640210 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.640805 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.641675 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.643157 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.643802 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.643945 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.644063 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.644240 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.644356 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:11:49.644543 12018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.640108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:53463 (local address 127.9.133.131:44731)
0807 06:11:49.640257 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.640262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.640518 (+ 256us) server_negotiation.cc:407] Connection header received
0807 06:11:49.640625 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.640630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.640691 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.640790 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.641113 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.641671 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.643301 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.643955 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.644082 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.644140 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.644318 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.644408 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:11:49.644512 12031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.639908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:44731 (local address 127.9.133.129:53463)
0807 06:11:49.640363 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.640405 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.640521 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.640822 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.640825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.640842 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.640977 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.640984 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.641833 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.641836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.643156 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.643166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.644071 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.644074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.644196 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.644219 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.644289 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.644336 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":45,"thread_start_us":128,"threads_started":1}
I20260807 06:11:49.644496 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.639318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:39667 (local address 127.9.133.129:56643)
0807 06:11:49.639694 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.639715 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.639868 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.640455 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.640457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.640500 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.640665 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.640673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.641811 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.641818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.642972 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.642982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.643937 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.643940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.644061 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.644121 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.644205 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.644303 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":92,"thread_start_us":163,"threads_started":1}
I20260807 06:11:49.645414 11930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87eb4c2fee184978b7ee060db534a6b7" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdb001829b742388fde98c74a3641f9" is_pre_election: true
I20260807 06:11:49.645422 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87eb4c2fee184978b7ee060db534a6b7" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dff49a14bb140809e083a035dc18f12" is_pre_election: true
I20260807 06:11:49.645802 11930 raft_consensus.cc:2454] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 0.
I20260807 06:11:49.645802 11729 raft_consensus.cc:2454] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 0.
I20260807 06:11:49.646353 11448 leader_election.cc:302] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdb001829b742388fde98c74a3641f9, c11c4c6e2d34422697f5245561a2839f; no voters:
I20260807 06:11:49.646623 12027 raft_consensus.cc:2781] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:49.646718 12027 raft_consensus.cc:482] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:49.646770 12027 raft_consensus.cc:3037] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:49.648123 12027 raft_consensus.cc:504] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.648509 12027 leader_election.cc:288] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 election: Requested vote from peers 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667), 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:49.648865 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87eb4c2fee184978b7ee060db534a6b7" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dff49a14bb140809e083a035dc18f12"
I20260807 06:11:49.648932 11930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87eb4c2fee184978b7ee060db534a6b7" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdb001829b742388fde98c74a3641f9"
I20260807 06:11:49.649027 11729 raft_consensus.cc:3037] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:49.649091 11930 raft_consensus.cc:3037] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:49.650460 11729 raft_consensus.cc:2454] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 1.
I20260807 06:11:49.650542 11930 raft_consensus.cc:2454] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 1.
I20260807 06:11:49.650931 11448 leader_election.cc:302] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdb001829b742388fde98c74a3641f9, c11c4c6e2d34422697f5245561a2839f; no voters:
I20260807 06:11:49.651130 12027 raft_consensus.cc:2781] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:49.651371 12027 raft_consensus.cc:686] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 1 LEADER]: Becoming Leader. State: Replica: c11c4c6e2d34422697f5245561a2839f, State: Running, Role: LEADER
I20260807 06:11:49.651836 12027 consensus_queue.cc:237] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.655882 11306 catalog_manager.cc:5495] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f reported cstate change: term changed from 0 to 1, leader changed from <none> to c11c4c6e2d34422697f5245561a2839f (127.9.133.129). New cstate: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260807 06:11:49.725070 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.720899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.9.133.129:45579)
0807 06:11:49.721061 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.721077 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.721095 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:11:49.721340 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.721343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.721417 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.721589 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:49.721868 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.722422 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.723385 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.723616 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.723695 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.724003 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:49.724560 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:49.724660 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.724816 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.724878 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 06:11:49.733072 11729 raft_consensus.cc:1261] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Refusing update from remote peer c11c4c6e2d34422697f5245561a2839f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:49.733889 12027 consensus_queue.cc:1035] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:49.734488 11930 raft_consensus.cc:1261] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Refusing update from remote peer c11c4c6e2d34422697f5245561a2839f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:49.735189 12027 consensus_queue.cc:1035] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:49.762828 11306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:49.763267 11306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:49.767061 11306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: Sending DeleteTablet for 3 replicas of tablet 87eb4c2fee184978b7ee060db534a6b7
I20260807 06:11:49.768260 11512 tablet_service.cc:1514] Processing DeleteTablet for tablet 87eb4c2fee184978b7ee060db534a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:49 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:46046
I20260807 06:11:49.768285 11896 tablet_service.cc:1514] Processing DeleteTablet for tablet 87eb4c2fee184978b7ee060db534a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:49 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:34360
I20260807 06:11:49.768918 12050 tablet_replica.cc:330] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: stopping tablet replica
I20260807 06:11:49.768935 11703 tablet_service.cc:1514] Processing DeleteTablet for tablet 87eb4c2fee184978b7ee060db534a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:49 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:55668
I20260807 06:11:49.769405 12049 tablet_replica.cc:330] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: stopping tablet replica
I20260807 06:11:49.769511 12050 raft_consensus.cc:2229] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:11:49.769671 12049 raft_consensus.cc:2229] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:49.770002 12049 raft_consensus.cc:2258] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:49.770195 12050 raft_consensus.cc:2258] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:49.770335 12051 tablet_replica.cc:330] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: stopping tablet replica
I20260807 06:11:49.770578 12051 raft_consensus.cc:2229] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:49.770800 12051 raft_consensus.cc:2258] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:49.772492 12051 ts_tablet_manager.cc:1905] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:49.772601 12050 ts_tablet_manager.cc:1905] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:49.773054 11306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:11:49.773630 11306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:49.775041 12049 ts_tablet_manager.cc:1905] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:49.776335 12050 ts_tablet_manager.cc:1918] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:11:49.776518 12050 log.cc:1196] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/87eb4c2fee184978b7ee060db534a6b7
I20260807 06:11:49.776942 12050 ts_tablet_manager.cc:1939] T 87eb4c2fee184978b7ee060db534a6b7 P c11c4c6e2d34422697f5245561a2839f: Deleting consensus metadata
I20260807 06:11:49.778393 11289 catalog_manager.cc:4841] TS c11c4c6e2d34422697f5245561a2839f (127.9.133.129:45579): tablet 87eb4c2fee184978b7ee060db534a6b7 (table test-table [id=c2fdbf2004a643548f0c83aca4ec8882]) successfully deleted
I20260807 06:11:49.779644 12051 ts_tablet_manager.cc:1918] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:11:49.779778 12051 log.cc:1196] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/87eb4c2fee184978b7ee060db534a6b7
I20260807 06:11:49.780139 12051 ts_tablet_manager.cc:1939] T 87eb4c2fee184978b7ee060db534a6b7 P 0dff49a14bb140809e083a035dc18f12: Deleting consensus metadata
I20260807 06:11:49.780860 11703 tablet_service.cc:1467] Processing CreateTablet for tablet 40aff336321246059e7081d876176eee (DEFAULT_TABLE table=test-table [id=fea0c5656eb64917b09fda5c70d504c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:49.780892 11896 tablet_service.cc:1467] Processing CreateTablet for tablet 40aff336321246059e7081d876176eee (DEFAULT_TABLE table=test-table [id=fea0c5656eb64917b09fda5c70d504c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:49.781154 11703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40aff336321246059e7081d876176eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:49.781490 11290 catalog_manager.cc:4841] TS 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667): tablet 87eb4c2fee184978b7ee060db534a6b7 (table test-table [id=c2fdbf2004a643548f0c83aca4ec8882]) successfully deleted
I20260807 06:11:49.782155 11896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40aff336321246059e7081d876176eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:49.783465 12049 ts_tablet_manager.cc:1918] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:11:49.783583 12049 log.cc:1196] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/87eb4c2fee184978b7ee060db534a6b7
I20260807 06:11:49.783732 12022 tablet_bootstrap.cc:492] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Bootstrap starting.
I20260807 06:11:49.784848 12021 tablet_bootstrap.cc:492] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Bootstrap starting.
I20260807 06:11:49.784924 12022 tablet_bootstrap.cc:654] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:49.786003 12021 tablet_bootstrap.cc:654] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:49.786541 11512 tablet_service.cc:1467] Processing CreateTablet for tablet 40aff336321246059e7081d876176eee (DEFAULT_TABLE table=test-table [id=fea0c5656eb64917b09fda5c70d504c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:49.786921 11512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40aff336321246059e7081d876176eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:49.787949 12021 tablet_bootstrap.cc:492] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: No bootstrap required, opened a new log
I20260807 06:11:49.788054 12021 ts_tablet_manager.cc:1397] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:11:49.788683 12021 raft_consensus.cc:348] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.788844 12021 raft_consensus.cc:374] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:49.788897 12021 raft_consensus.cc:729] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdb001829b742388fde98c74a3641f9, State: Initialized, Role: FOLLOWER
I20260807 06:11:49.789049 12021 consensus_queue.cc:260] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.789343 12021 ts_tablet_manager.cc:1428] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 06:11:49.790323 12054 tablet_bootstrap.cc:492] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Bootstrap starting.
I20260807 06:11:49.790803 12022 tablet_bootstrap.cc:492] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: No bootstrap required, opened a new log
I20260807 06:11:49.790911 12022 ts_tablet_manager.cc:1397] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 06:11:49.791467 12022 raft_consensus.cc:348] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.791630 12022 raft_consensus.cc:374] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:49.791676 12022 raft_consensus.cc:729] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dff49a14bb140809e083a035dc18f12, State: Initialized, Role: FOLLOWER
I20260807 06:11:49.784837 12049 ts_tablet_manager.cc:1939] T 87eb4c2fee184978b7ee060db534a6b7 P 2fdb001829b742388fde98c74a3641f9: Deleting consensus metadata
I20260807 06:11:49.791846 12022 consensus_queue.cc:260] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.792155 12022 ts_tablet_manager.cc:1428] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:11:49.792271 12054 tablet_bootstrap.cc:654] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:49.794090 11290 catalog_manager.cc:4841] TS 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731): tablet 87eb4c2fee184978b7ee060db534a6b7 (table test-table [id=c2fdbf2004a643548f0c83aca4ec8882]) successfully deleted
I20260807 06:11:49.795111 12054 tablet_bootstrap.cc:492] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: No bootstrap required, opened a new log
I20260807 06:11:49.795223 12054 ts_tablet_manager.cc:1397] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 06:11:49.795707 12054 raft_consensus.cc:348] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.795830 12054 raft_consensus.cc:374] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:49.795861 12054 raft_consensus.cc:729] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c11c4c6e2d34422697f5245561a2839f, State: Initialized, Role: FOLLOWER
I20260807 06:11:49.796012 12054 consensus_queue.cc:260] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.796309 12054 ts_tablet_manager.cc:1428] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:11:49.993369 12028 raft_consensus.cc:482] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:49.993613 12028 raft_consensus.cc:504] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:49.995312 12028 leader_election.cc:288] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c11c4c6e2d34422697f5245561a2839f (127.9.133.129:45579), 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:49.999858 12018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.995781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:51517 (local address 127.9.133.131:44731)
0807 06:11:49.996055 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.996062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.996161 (+ 99us) server_negotiation.cc:407] Connection header received
0807 06:11:49.996379 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.996382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.996433 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.996534 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.996945 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.997462 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.998659 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.999163 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.999295 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.999367 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.999589 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.999690 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260807 06:11:49.999879 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.995712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34637 (local address 127.9.133.129:45579)
0807 06:11:49.995892 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:49.995898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:49.996142 (+ 244us) server_negotiation.cc:407] Connection header received
0807 06:11:49.996379 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:49.996382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:49.996436 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:49.996545 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:49.996895 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.997463 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.998653 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.999169 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.999297 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.999541 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:49.999682 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 06:11:49.999758 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:11:49.999954 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.995602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:45579 (local address 127.9.133.130:34637)
0807 06:11:49.996003 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.996019 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.996239 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.996559 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.996562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.996591 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.996779 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.996786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.997640 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.997643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.998521 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.998529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.999295 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.999299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.999548 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.999575 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.999662 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.999724 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":51,"spinlock_wait_cycles":58624,"thread_start_us":133,"threads_started":1}
I20260807 06:11:50.000176 12056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:49.995602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:44731 (local address 127.9.133.130:51517)
0807 06:11:49.996022 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:49.996035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:49.996238 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:49.996560 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:49.996562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:49.996593 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:49.996798 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.996803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.997576 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.997580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.998519 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:49.998529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:49.999295 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:49.999299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:49.999427 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:49.999457 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 06:11:49.999540 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 06:11:49.999629 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":130,"spinlock_wait_cycles":22528,"thread_start_us":113,"threads_started":1}
I20260807 06:11:50.000766 11930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40aff336321246059e7081d876176eee" candidate_uuid: "0dff49a14bb140809e083a035dc18f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdb001829b742388fde98c74a3641f9" is_pre_election: true
I20260807 06:11:50.000835 11538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40aff336321246059e7081d876176eee" candidate_uuid: "0dff49a14bb140809e083a035dc18f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c11c4c6e2d34422697f5245561a2839f" is_pre_election: true
I20260807 06:11:50.001025 11930 raft_consensus.cc:2454] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dff49a14bb140809e083a035dc18f12 in term 0.
I20260807 06:11:50.001174 11538 raft_consensus.cc:2454] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dff49a14bb140809e083a035dc18f12 in term 0.
I20260807 06:11:50.001588 11645 leader_election.cc:302] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dff49a14bb140809e083a035dc18f12, 2fdb001829b742388fde98c74a3641f9; no voters:
I20260807 06:11:50.002033 12028 raft_consensus.cc:2781] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:50.002099 12028 raft_consensus.cc:482] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:50.002128 12028 raft_consensus.cc:3037] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:50.003690 12028 raft_consensus.cc:504] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.004164 12028 leader_election.cc:288] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [CANDIDATE]: Term 1 election: Requested vote from peers c11c4c6e2d34422697f5245561a2839f (127.9.133.129:45579), 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:50.004777 11930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40aff336321246059e7081d876176eee" candidate_uuid: "0dff49a14bb140809e083a035dc18f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdb001829b742388fde98c74a3641f9"
I20260807 06:11:50.004837 11538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40aff336321246059e7081d876176eee" candidate_uuid: "0dff49a14bb140809e083a035dc18f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c11c4c6e2d34422697f5245561a2839f"
I20260807 06:11:50.004988 11930 raft_consensus.cc:3037] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:50.005091 11538 raft_consensus.cc:3037] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:50.006253 11538 raft_consensus.cc:2454] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dff49a14bb140809e083a035dc18f12 in term 1.
I20260807 06:11:50.006258 11930 raft_consensus.cc:2454] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dff49a14bb140809e083a035dc18f12 in term 1.
I20260807 06:11:50.006678 11645 leader_election.cc:302] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dff49a14bb140809e083a035dc18f12, 2fdb001829b742388fde98c74a3641f9; no voters:
I20260807 06:11:50.007164 12028 raft_consensus.cc:2781] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:50.007529 12028 raft_consensus.cc:686] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 1 LEADER]: Becoming Leader. State: Replica: 0dff49a14bb140809e083a035dc18f12, State: Running, Role: LEADER
I20260807 06:11:50.007942 12028 consensus_queue.cc:237] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.010942 11306 catalog_manager.cc:5495] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dff49a14bb140809e083a035dc18f12 (127.9.133.130). New cstate: current_term: 1 leader_uuid: "0dff49a14bb140809e083a035dc18f12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } health_report { overall_health: UNKNOWN } } }
Aug 07 06:11:50 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:50.064154 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.053665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.9.133.190:42915)
0807 06:11:50.054026 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.054037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.054138 (+ 101us) server_negotiation.cc:407] Connection header received
0807 06:11:50.054311 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.054314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.054368 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.054469 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:50.055403 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.056004 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.056803 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.057029 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.057111 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.060157 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:50.060161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:50.060163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:50.060190 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:50.061898 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:50.062500 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:50.062503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:50.062505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:50.062606 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:50.063065 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:50.063072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:50.063077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:50.063339 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:50.063443 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.063828 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.063971 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":158,"threads_started":1}
I20260807 06:11:50.064154 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.053604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42915 (local address 127.0.0.1:34708)
0807 06:11:50.054010 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:50.054024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:50.054175 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:50.054511 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:50.054513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:50.055149 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:50.055277 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.055284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.056111 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.056114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.056703 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.056710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.057140 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.057142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.057208 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.057892 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:50.057905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:50.059951 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:50.062047 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:50.062059 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:50.062068 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:50.062379 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:50.062740 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:50.062743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:50.062745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:50.062892 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:50.063483 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:50.063492 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:50.063631 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 06:11:50.063742 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:11:50.063904 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":139,"threads_started":1}
I20260807 06:11:50.070147 11306 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:34708:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:11:50.071257 11306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:50.077521 11512 tablet_service.cc:1467] Processing CreateTablet for tablet 376c1492cb1542fdbb88e6411f9b06a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=030b2b3892e2419ba5db3b895c73dcc8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:11:50.077570 11703 tablet_service.cc:1467] Processing CreateTablet for tablet 376c1492cb1542fdbb88e6411f9b06a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=030b2b3892e2419ba5db3b895c73dcc8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:11:50.077585 11896 tablet_service.cc:1467] Processing CreateTablet for tablet 376c1492cb1542fdbb88e6411f9b06a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=030b2b3892e2419ba5db3b895c73dcc8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:11:50.078173 11512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 376c1492cb1542fdbb88e6411f9b06a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:50.078168 11896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 376c1492cb1542fdbb88e6411f9b06a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:50.078168 11703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 376c1492cb1542fdbb88e6411f9b06a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:50.081440 12054 tablet_bootstrap.cc:492] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: Bootstrap starting.
I20260807 06:11:50.081918 12021 tablet_bootstrap.cc:492] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: Bootstrap starting.
I20260807 06:11:50.082688 12054 tablet_bootstrap.cc:654] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:50.082983 12021 tablet_bootstrap.cc:654] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:50.083588 12022 tablet_bootstrap.cc:492] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: Bootstrap starting.
I20260807 06:11:50.084136 12054 tablet_bootstrap.cc:492] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: No bootstrap required, opened a new log
I20260807 06:11:50.084252 12054 ts_tablet_manager.cc:1397] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:11:50.084599 12021 tablet_bootstrap.cc:492] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: No bootstrap required, opened a new log
I20260807 06:11:50.084692 12021 ts_tablet_manager.cc:1397] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 06:11:50.084772 12022 tablet_bootstrap.cc:654] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:50.084765 12054 raft_consensus.cc:348] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.084906 12054 raft_consensus.cc:374] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:50.084950 12054 raft_consensus.cc:729] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c11c4c6e2d34422697f5245561a2839f, State: Initialized, Role: FOLLOWER
I20260807 06:11:50.085229 12021 raft_consensus.cc:348] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.085384 12021 raft_consensus.cc:374] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:50.085427 12021 raft_consensus.cc:729] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdb001829b742388fde98c74a3641f9, State: Initialized, Role: FOLLOWER
I20260807 06:11:50.085487 12054 consensus_queue.cc:260] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.085587 12021 consensus_queue.cc:260] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.085863 12054 ts_tablet_manager.cc:1428] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:11:50.085923 12021 ts_tablet_manager.cc:1428] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:11:50.085916 12034 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.086097 12034 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.085927 12029 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.086699 12029 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.086958 12022 tablet_bootstrap.cc:492] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: No bootstrap required, opened a new log
I20260807 06:11:50.087114 12022 ts_tablet_manager.cc:1397] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 06:11:50.087754 12022 raft_consensus.cc:348] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.087894 12022 raft_consensus.cc:374] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:50.087942 12022 raft_consensus.cc:729] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dff49a14bb140809e083a035dc18f12, State: Initialized, Role: FOLLOWER
I20260807 06:11:50.088094 12022 consensus_queue.cc:260] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.088363 12028 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.088501 12028 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.088785 12022 ts_tablet_manager.cc:1428] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 06:11:50.159214 12034 raft_consensus.cc:482] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:50.159433 12034 raft_consensus.cc:504] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.159955 12034 leader_election.cc:288] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667), 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:50.160670 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c1492cb1542fdbb88e6411f9b06a3" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dff49a14bb140809e083a035dc18f12" is_pre_election: true
I20260807 06:11:50.160701 11930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c1492cb1542fdbb88e6411f9b06a3" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdb001829b742388fde98c74a3641f9" is_pre_election: true
I20260807 06:11:50.160876 11930 raft_consensus.cc:2454] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 0.
I20260807 06:11:50.160876 11729 raft_consensus.cc:2454] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 0.
I20260807 06:11:50.161259 11448 leader_election.cc:302] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dff49a14bb140809e083a035dc18f12, c11c4c6e2d34422697f5245561a2839f; no voters:
I20260807 06:11:50.161512 12034 raft_consensus.cc:2781] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:50.161592 12034 raft_consensus.cc:482] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:50.161634 12034 raft_consensus.cc:3037] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:50.162976 12034 raft_consensus.cc:504] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.163439 12034 leader_election.cc:288] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 election: Requested vote from peers 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667), 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731)
I20260807 06:11:50.163837 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c1492cb1542fdbb88e6411f9b06a3" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dff49a14bb140809e083a035dc18f12"
I20260807 06:11:50.163908 11930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376c1492cb1542fdbb88e6411f9b06a3" candidate_uuid: "c11c4c6e2d34422697f5245561a2839f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdb001829b742388fde98c74a3641f9"
I20260807 06:11:50.164049 11930 raft_consensus.cc:3037] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:50.164052 11729 raft_consensus.cc:3037] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:50.165433 11729 raft_consensus.cc:2454] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 1.
I20260807 06:11:50.165436 11930 raft_consensus.cc:2454] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c11c4c6e2d34422697f5245561a2839f in term 1.
I20260807 06:11:50.165776 11448 leader_election.cc:302] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdb001829b742388fde98c74a3641f9, c11c4c6e2d34422697f5245561a2839f; no voters:
I20260807 06:11:50.165969 12034 raft_consensus.cc:2781] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:50.166245 12034 raft_consensus.cc:686] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [term 1 LEADER]: Becoming Leader. State: Replica: c11c4c6e2d34422697f5245561a2839f, State: Running, Role: LEADER
I20260807 06:11:50.166462 12034 consensus_queue.cc:237] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } }
I20260807 06:11:50.168232 12027 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: TxnStatusTablet state changed. Reason: New leader c11c4c6e2d34422697f5245561a2839f. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.168370 12027 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:11:50.168751 12077 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:11:50.169016 12077 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:11:50.169312 11299 catalog_manager.cc:5495] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f reported cstate change: term changed from 0 to 1, leader changed from <none> to c11c4c6e2d34422697f5245561a2839f (127.9.133.129). New cstate: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260807 06:11:50.195325 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.191605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.9.133.129:45579)
0807 06:11:50.191759 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.191765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.191852 (+ 87us) server_negotiation.cc:407] Connection header received
0807 06:11:50.192088 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.192092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.192165 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.192254 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:50.192553 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.193144 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.194078 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.194331 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.194445 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.194769 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.194868 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.194961 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.195095 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.195155 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:11:50.195325 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.191492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:45579 (local address 127.0.0.1:46056)
0807 06:11:50.191781 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:50.191798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:50.191929 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:50.192279 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:50.192283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:50.192291 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:50.192441 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.192447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.193266 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.193270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.193943 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.193952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.194453 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.194457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.194568 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.194619 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.194967 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.194988 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 06:11:50.195075 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:11:50.195179 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260807 06:11:50.198634 11930 raft_consensus.cc:1261] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Refusing update from remote peer c11c4c6e2d34422697f5245561a2839f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:50.198747 11729 raft_consensus.cc:1261] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Refusing update from remote peer c11c4c6e2d34422697f5245561a2839f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:50.199210 12034 consensus_queue.cc:1035] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:50.199380 12027 consensus_queue.cc:1035] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:50.201002 12028 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: TxnStatusTablet state changed. Reason: New leader c11c4c6e2d34422697f5245561a2839f. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.201143 12028 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.201174 12029 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: TxnStatusTablet state changed. Reason: New leader c11c4c6e2d34422697f5245561a2839f. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.201303 12029 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.202862 12027 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.203852 12040 mvcc.cc:205] Tried to move back new op lower bound from 7315796419367886848 to 7315796419243638784. Current Snapshot: MvccSnapshot[applied={T|T < 7315796419367886848}]
I20260807 06:11:50.202992 12027 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:11:50.204483 12029 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.204588 12029 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P 2fdb001829b742388fde98c74a3641f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.203214 12037 mvcc.cc:205] Tried to move back new op lower bound from 7315796419367886848 to 7315796419243638784. Current Snapshot: MvccSnapshot[applied={T|T < 7315796419367886848 or (T in {7315796419367886848})}]
I20260807 06:11:50.205140 12038 mvcc.cc:205] Tried to move back new op lower bound from 7315796419367886848 to 7315796419243638784. Current Snapshot: MvccSnapshot[applied={T|T < 7315796419367886848}]
I20260807 06:11:50.205304 12028 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.205399 12028 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P 0dff49a14bb140809e083a035dc18f12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:50.206666 12027 tablet_replica.cc:439] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c11c4c6e2d34422697f5245561a2839f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 } } peers { permanent_uuid: "0dff49a14bb140809e083a035dc18f12" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 39667 } } peers { permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 } } }
I20260807 06:11:50.206769 12027 tablet_replica.cc:442] T 376c1492cb1542fdbb88e6411f9b06a3 P c11c4c6e2d34422697f5245561a2839f: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:11:50.219187 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.214069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.9.133.130:39667)
0807 06:11:50.214442 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.214451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.214464 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:11:50.214550 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.214553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.214615 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.214728 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:50.215059 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.215680 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.216913 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.217213 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.217320 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.217662 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.218286 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.218413 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.218530 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.218599 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":176,"threads_started":1}
I20260807 06:11:50.234331 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.229609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:40771 (local address 127.9.133.129:45579)
0807 06:11:50.230302 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.230307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.230324 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:11:50.230403 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.230406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.230471 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.230551 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:50.231073 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.231876 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.233214 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.233435 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.233526 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.233864 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.233947 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.234005 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.234128 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.234188 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260807 06:11:50.234376 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.229360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:45579 (local address 127.9.133.130:40771)
0807 06:11:50.229830 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:50.229850 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:50.229996 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:50.230699 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:50.230703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:50.230713 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:50.230875 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.230881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.232004 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.232007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.233086 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.233097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.233550 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.233554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.233658 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.233720 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.234032 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.234039 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:11:50.234109 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:11:50.234192 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":169,"threads_started":1}
I20260807 06:11:50.243269 11930 raft_consensus.cc:1261] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Refusing update from remote peer 0dff49a14bb140809e083a035dc18f12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:50.243651 11538 raft_consensus.cc:1261] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Refusing update from remote peer 0dff49a14bb140809e083a035dc18f12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:50.243844 12058 consensus_queue.cc:1035] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdb001829b742388fde98c74a3641f9" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:50.244227 12028 consensus_queue.cc:1035] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11c4c6e2d34422697f5245561a2839f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 45579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:50.272559 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.268187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:37211 (local address 127.9.133.130:39667)
0807 06:11:50.268524 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.268529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.268681 (+ 152us) server_negotiation.cc:407] Connection header received
0807 06:11:50.268877 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.268881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.268944 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.269039 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:50.269512 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.270350 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.271297 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.271645 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.271748 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.272024 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.272121 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.272194 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.272329 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.272391 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260807 06:11:50.272567 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.268070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:39667 (local address 127.9.133.129:37211)
0807 06:11:50.268541 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:50.268559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:50.268745 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:50.269243 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:50.269247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:50.269256 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:50.269392 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.269399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.270449 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.270452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.271185 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:50.271195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.271761 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.271764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:50.271834 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.271884 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:50.272209 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:50.272221 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 06:11:50.272299 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:11:50.272396 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":196,"threads_started":1}
I20260807 06:11:50.304805 11299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:50.305047 11299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:50.308259 11299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d3b2b282dae477b915f8be542223429: Sending DeleteTablet for 3 replicas of tablet 40aff336321246059e7081d876176eee
I20260807 06:11:50.309141 11512 tablet_service.cc:1514] Processing DeleteTablet for tablet 40aff336321246059e7081d876176eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:50 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:46046
I20260807 06:11:50.309696 11703 tablet_service.cc:1514] Processing DeleteTablet for tablet 40aff336321246059e7081d876176eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:50 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:55668
I20260807 06:11:50.310067 11896 tablet_service.cc:1514] Processing DeleteTablet for tablet 40aff336321246059e7081d876176eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:50 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:34360
I20260807 06:11:50.312244 12102 tablet_replica.cc:330] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: stopping tablet replica
I20260807 06:11:50.312505 12104 tablet_replica.cc:330] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: stopping tablet replica
I20260807 06:11:50.312747 12104 raft_consensus.cc:2229] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:50.312994 12102 raft_consensus.cc:2229] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:11:50.313235 12104 raft_consensus.cc:2258] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:50.313680 12102 raft_consensus.cc:2258] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:50.314698 12104 ts_tablet_manager.cc:1905] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:50.315558 12102 ts_tablet_manager.cc:1905] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:50.318184 12100 tablet_replica.cc:330] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: stopping tablet replica
I20260807 06:11:50.318347 12100 raft_consensus.cc:2229] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:50.318497 12100 raft_consensus.cc:2258] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:50.319375 12100 ts_tablet_manager.cc:1905] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:50.322805 12100 ts_tablet_manager.cc:1918] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:50.322908 12100 log.cc:1196] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/40aff336321246059e7081d876176eee
I20260807 06:11:50.323302 12100 ts_tablet_manager.cc:1939] T 40aff336321246059e7081d876176eee P c11c4c6e2d34422697f5245561a2839f: Deleting consensus metadata
I20260807 06:11:50.324245 11289 catalog_manager.cc:4841] TS c11c4c6e2d34422697f5245561a2839f (127.9.133.129:45579): tablet 40aff336321246059e7081d876176eee (table test-table [id=fea0c5656eb64917b09fda5c70d504c5]) successfully deleted
I20260807 06:11:50.324761 12102 ts_tablet_manager.cc:1918] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:50.324856 12102 log.cc:1196] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/40aff336321246059e7081d876176eee
I20260807 06:11:50.325140 12102 ts_tablet_manager.cc:1939] T 40aff336321246059e7081d876176eee P 0dff49a14bb140809e083a035dc18f12: Deleting consensus metadata
I20260807 06:11:50.325855 12104 ts_tablet_manager.cc:1918] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:50.325956 12104 log.cc:1196] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/40aff336321246059e7081d876176eee
I20260807 06:11:50.326036 11290 catalog_manager.cc:4841] TS 0dff49a14bb140809e083a035dc18f12 (127.9.133.130:39667): tablet 40aff336321246059e7081d876176eee (table test-table [id=fea0c5656eb64917b09fda5c70d504c5]) successfully deleted
I20260807 06:11:50.326243 12104 ts_tablet_manager.cc:1939] T 40aff336321246059e7081d876176eee P 2fdb001829b742388fde98c74a3641f9: Deleting consensus metadata
I20260807 06:11:50.327119 11290 catalog_manager.cc:4841] TS 2fdb001829b742388fde98c74a3641f9 (127.9.133.131:44731): tablet 40aff336321246059e7081d876176eee (table test-table [id=fea0c5656eb64917b09fda5c70d504c5]) successfully deleted
Aug 07 06:11:50 dist-test-slave-h3ft krb5kdc[11245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:11:50.340811 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.316138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.9.133.129:45579)
0807 06:11:50.316287 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.316293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.327331 (+ 11038us) server_negotiation.cc:407] Connection header received
0807 06:11:50.327585 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.327589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.327657 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.327759 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:50.328825 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.329669 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.330941 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.331333 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.331437 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.335761 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:50.335780 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:50.335807 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:50.335869 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:50.338289 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:50.338830 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:50.338840 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:50.338842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:50.338952 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:50.339360 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:50.339363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:50.339365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:50.339783 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:50.340043 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.340235 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.340421 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 06:11:50.341488 11602 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46058
I20260807 06:11:50.357148 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:50.344134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.9.133.190:42915)
0807 06:11:50.344289 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:50.344294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:50.347447 (+ 3153us) server_negotiation.cc:407] Connection header received
0807 06:11:50.347583 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:50.347586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:50.347654 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:50.347746 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:50.348642 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.349564 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.350849 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:50.351286 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:50.351390 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:50.353255 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:50.353260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:50.353263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:50.353295 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:50.355264 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:50.355745 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:50.355749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:50.355750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:50.355851 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:50.356157 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:50.356160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:50.356162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:50.356393 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:50.356533 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:50.356804 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 06:11:50.356952 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 06:11:50.357815 11299 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34712
I20260807 06:11:50.360400 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 11426
I20260807 06:11:50.375545 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 11618
I20260807 06:11:50.389248 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 11809
I20260807 06:11:50.401384 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 11265
2026-08-07T06:11:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3781 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:11:50 dist-test-slave-h3ft krb5kdc[12115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:11:50 dist-test-slave-h3ft krb5kdc[12115](info): set up 2 sockets
Aug 07 06:11:50 dist-test-slave-h3ft krb5kdc[12115](info): commencing operation
krb5kdc: starting...
W20260807 06:11:51.700094 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:11:51 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:11:51.850466 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45551
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:51.982349 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:51.982795 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:51.982887 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:51.987222 12130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:11:51.987346 12130 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:11:51.987377 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:51.987412 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:51.987473 12130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:11:51.987514 12130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:11:51.993443 12130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:45551
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:51.994915 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:51.996287 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:52.006546 12138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.006567 12139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.006567 12142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:52.006726 12130 server_base.cc:1034] running on GCE node
I20260807 06:11:52.007681 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:52.009065 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:52.010272 12130 hybrid_clock.cc:648] HybridClock initialized: now 1786083112010255 us; error 31 us; skew 500 ppm
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:52.013739 12130 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:11:52.014884 12130 webserver.cc:460] Webserver started at http://127.9.133.190:34385/ using document root <none> and password file <none>
I20260807 06:11:52.015657 12130 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:52.015763 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:52.016053 12130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:52.018635 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "4dc3d4ecade1433eb2646fce50ba54ec"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "aca3af8fd69a9b2c321a214a829089a8"
server_key_iv: "8a735c45efc2bc2271983035fa51de47"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.019379 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "4dc3d4ecade1433eb2646fce50ba54ec"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "aca3af8fd69a9b2c321a214a829089a8"
server_key_iv: "8a735c45efc2bc2271983035fa51de47"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.024353 12130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 06:11:52.027426 12152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.028964 12130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 06:11:52.029136 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "4dc3d4ecade1433eb2646fce50ba54ec"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "aca3af8fd69a9b2c321a214a829089a8"
server_key_iv: "8a735c45efc2bc2271983035fa51de47"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.029275 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:52.054623 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:52.058571 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:52.058832 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:52.069016 12130 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:45551
I20260807 06:11:52.069052 12235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:45551 every 8 connection(s)
I20260807 06:11:52.070214 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:11:52.074108 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:52.079829 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 12130
I20260807 06:11:52.080008 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:11:52.080370 9750 external_mini_cluster.cc:1428] Setting key 868985a5fcb0b10618300b60a8baa382
I20260807 06:11:52.082618 12237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Bootstrap starting.
I20260807 06:11:52.085759 12237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:52.086759 12237 log.cc:824] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:52.089273 12237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: No bootstrap required, opened a new log
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:52.093869 12237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45551 } }
I20260807 06:11:52.094201 12237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:52.094287 12237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc3d4ecade1433eb2646fce50ba54ec, State: Initialized, Role: FOLLOWER
I20260807 06:11:52.094955 12237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45551 } }
I20260807 06:11:52.095192 12237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:11:52.095270 12237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:11:52.095396 12237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:52.096728 12237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45551 } }
I20260807 06:11:52.097179 12237 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dc3d4ecade1433eb2646fce50ba54ec; no voters:
I20260807 06:11:52.097532 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.082446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.9.133.190:45551)
0807 06:11:52.083079 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.083099 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.083145 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:11:52.083923 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.083944 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.084360 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.084771 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.085777 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.087111 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.088060 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.088596 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.088786 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.091740 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.091747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.091764 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.091796 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.093756 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.094259 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.094269 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.094277 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.094502 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.094812 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.094817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.094819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.095528 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.095755 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.096108 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.096448 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":185,"threads_started":1}
I20260807 06:11:52.097661 12237 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:11:52.097761 12245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:52.097997 12245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [term 1 LEADER]: Becoming Leader. State: Replica: 4dc3d4ecade1433eb2646fce50ba54ec, State: Running, Role: LEADER
I20260807 06:11:52.098377 12245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45551 } }
I20260807 06:11:52.099424 12237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:11:52.100550 12247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45551 } } }
I20260807 06:11:52.100754 12247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:52.101256 12248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dc3d4ecade1433eb2646fce50ba54ec. Latest consensus state: current_term: 1 leader_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc3d4ecade1433eb2646fce50ba54ec" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45551 } } }
I20260807 06:11:52.101348 12258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:11:52.101372 12248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec [sys.catalog]: This master's current role is: LEADER
I20260807 06:11:52.105746 12258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:11:52.113894 12258 catalog_manager.cc:1347] Generated new cluster ID: 1ab8ce573c0b476887c951863369bb4d
I20260807 06:11:52.114024 12258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:11:52.144263 12258 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:11:52.145846 12258 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:11:52.155913 12258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Generated new TSK 0
I20260807 06:11:52.156834 12258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:11:52.198968 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45551
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:52.353085 12275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:52.353394 12275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:52.353442 12275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:52.357643 12275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:52.357709 12275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:52.357750 12275 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:11:52.357793 12275 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:11:52.357854 12275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:11:52.363493 12275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:52.364889 12275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:52.366209 12275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:52.378005 12284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:52.378253 12275 server_base.cc:1034] running on GCE node
W20260807 06:11:52.378001 12282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.378036 12281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:52.378913 12275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:52.379782 12275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:52.380950 12275 hybrid_clock.cc:648] HybridClock initialized: now 1786083112380935 us; error 27 us; skew 500 ppm
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:52.384258 12275 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:11:52.385314 12275 webserver.cc:460] Webserver started at http://127.9.133.129:34725/ using document root <none> and password file <none>
I20260807 06:11:52.385954 12275 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:52.386020 12275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:52.386212 12275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:52.388608 12275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "2844b4bcc165412dbf726ec23fa1fc55"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "ae5c94b66c844a92c328b74a361a063e"
server_key_iv: "b967a4ab024437c6ac28ed8f3fbb70a0"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.389314 12275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "2844b4bcc165412dbf726ec23fa1fc55"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "ae5c94b66c844a92c328b74a361a063e"
server_key_iv: "b967a4ab024437c6ac28ed8f3fbb70a0"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.393798 12275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 06:11:52.396670 12293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.398123 12275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:11:52.398255 12275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "2844b4bcc165412dbf726ec23fa1fc55"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "ae5c94b66c844a92c328b74a361a063e"
server_key_iv: "b967a4ab024437c6ac28ed8f3fbb70a0"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.398368 12275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:52.412101 12275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:52.416191 12275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:52.416429 12275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:52.418267 12275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:52.418321 12275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.418370 12275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:52.418416 12275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:52.431777 12275 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:39095
I20260807 06:11:52.431813 12450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:39095 every 8 connection(s)
I20260807 06:11:52.433198 12275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:11:52.434697 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.419308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:48439 (local address 127.9.133.190:45551)
0807 06:11:52.419474 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.419478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.420721 (+ 1243us) server_negotiation.cc:407] Connection header received
0807 06:11:52.422101 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.422107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.422199 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.422302 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.423892 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.424608 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.426038 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.426303 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.426431 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.430050 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.430055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.430058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.430101 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.432016 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.432638 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.432641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.432642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.432728 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.433165 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.433168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.433169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.433482 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.433605 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.434332 (+ 727us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.434489 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:11:52.435735 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.419909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.9.133.129:48439)
0807 06:11:52.420530 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:52.420579 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:52.421778 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:52.422553 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:52.422561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:52.422947 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:52.423668 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.423697 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.424757 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.424760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.425890 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.425901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.426499 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.426504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.426696 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.427552 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:52.427580 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:52.429781 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:52.432146 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.432152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.432168 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.432527 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.432839 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.432842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.432845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.433055 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.433613 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:52.433621 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:52.434011 (+ 390us) client_negotiation.cc:769] Sending connection context
0807 06:11:52.434241 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 06:11:52.434648 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":166,"threads_started":1}
I20260807 06:11:52.437176 12452 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45551
I20260807 06:11:52.437364 12452 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:52.438014 12452 heartbeater.cc:511] Master 127.9.133.190:45551 requested a full tablet report, sending...
I20260807 06:11:52.440091 12177 ts_manager.cc:194] Registered new tserver with Master: 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095)
I20260807 06:11:52.441641 12177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:48439
I20260807 06:11:52.442696 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 12275
I20260807 06:11:52.442853 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:11:52.443252 9750 external_mini_cluster.cc:1428] Setting key 8476be9c46ae60b8e9029d601c302c14
I20260807 06:11:52.448334 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.439853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:53947 (local address 127.9.133.190:45551)
0807 06:11:52.440007 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.440011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.440242 (+ 231us) server_negotiation.cc:407] Connection header received
0807 06:11:52.440367 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.440370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.440426 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.440502 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.441116 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.441657 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.442789 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.443159 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.443265 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.444826 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.444830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.444833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.444860 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.446440 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.446981 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.446985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.446987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.447123 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.447448 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.447450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.447452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.447637 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.447761 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.448037 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.448163 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:11:52.448352 12459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.439779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.9.133.129:53947)
0807 06:11:52.440139 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:52.440156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:52.440272 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:52.440543 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:52.440546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:52.440801 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:52.440998 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.441007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.441792 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.441796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.442623 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.442635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.443307 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.443311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.443410 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.444173 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:52.444188 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:52.444672 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:52.446563 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.446570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.446575 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.446869 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.447217 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.447224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.447229 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.447351 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.447748 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:52.447751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:52.447853 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 06:11:52.447967 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 06:11:52.448144 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:11:52.479542 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45551
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:52.612947 12463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:52.613293 12463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:52.613389 12463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:52.617734 12463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:52.617803 12463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:52.617880 12463 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:11:52.617910 12463 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:11:52.617965 12463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:11:52.623786 12463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:52.625162 12463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:52.626457 12463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:52.637877 12471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.637887 12473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.637955 12476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:52.638257 12463 server_base.cc:1034] running on GCE node
I20260807 06:11:52.638767 12463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:52.639571 12463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:52.640760 12463 hybrid_clock.cc:648] HybridClock initialized: now 1786083112640728 us; error 43 us; skew 500 ppm
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:52.644294 12463 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:11:52.645449 12463 webserver.cc:460] Webserver started at http://127.9.133.130:34101/ using document root <none> and password file <none>
I20260807 06:11:52.646212 12463 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:52.646313 12463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:52.646595 12463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:52.649180 12463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "267ff4b474c748e797e450485f1dd0e7"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "8887df1bdb76834f7dee7547f6707b64"
server_key_iv: "2cc1e67aaab5a25a2b494e93545220c0"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.649886 12463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "267ff4b474c748e797e450485f1dd0e7"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "8887df1bdb76834f7dee7547f6707b64"
server_key_iv: "2cc1e67aaab5a25a2b494e93545220c0"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.654461 12463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 06:11:52.657359 12485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.658932 12463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 06:11:52.659133 12463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "267ff4b474c748e797e450485f1dd0e7"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "8887df1bdb76834f7dee7547f6707b64"
server_key_iv: "2cc1e67aaab5a25a2b494e93545220c0"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.659271 12463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:52.677203 12463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:52.681420 12463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:52.681665 12463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:52.683590 12463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:52.683670 12463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.683745 12463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:52.683775 12463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:52.699128 12463 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:43583
I20260807 06:11:52.699149 12649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:43583 every 8 connection(s)
I20260807 06:11:52.700400 12463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:11:52.702185 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.684383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:55965 (local address 127.9.133.190:45551)
0807 06:11:52.684587 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.684593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.686145 (+ 1552us) server_negotiation.cc:407] Connection header received
0807 06:11:52.687885 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.687889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.687954 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.688036 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.690340 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.691197 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.693236 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.693491 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.693587 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.697325 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.697331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.697336 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.697377 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.699455 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.700074 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.700078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.700081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.700191 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.700647 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.700651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.700653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.700985 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.701100 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.701868 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.702011 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 06:11:52.703330 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.685347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.9.133.130:55965)
0807 06:11:52.685923 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:52.685999 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:52.687572 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:52.688530 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:52.688544 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:52.689186 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:52.690120 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.690152 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.691384 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.691389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.693092 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.693107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.693678 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.693686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.693956 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.694796 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:52.694823 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:52.697104 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:52.699595 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.699602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.699621 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.699941 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.700294 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.700297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.700298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.700549 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.701158 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:52.701166 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:52.701588 (+ 422us) client_negotiation.cc:769] Sending connection context
0807 06:11:52.701825 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 06:11:52.702198 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":128,"threads_started":1}
I20260807 06:11:52.704795 12651 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45551
I20260807 06:11:52.704962 12651 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:52.705672 12651 heartbeater.cc:511] Master 127.9.133.190:45551 requested a full tablet report, sending...
I20260807 06:11:52.707108 12178 ts_manager.cc:194] Registered new tserver with Master: 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583)
I20260807 06:11:52.708123 12178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:55965
I20260807 06:11:52.708760 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 12463
I20260807 06:11:52.708923 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:11:52.709352 9750 external_mini_cluster.cc:1428] Setting key a2adf531f15ca96557c45f6ddc5a514e
I20260807 06:11:52.716185 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.707775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:59423 (local address 127.9.133.190:45551)
0807 06:11:52.707907 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.707910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.708146 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:11:52.708297 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.708300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.708353 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.708439 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.709331 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.710007 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.710860 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.711186 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.711293 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.712603 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.712610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.712616 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.712650 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.714335 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.714798 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.714803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.714805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.714898 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.715240 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.715244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.715246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.715494 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.715607 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.715925 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.716046 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 06:11:52.716322 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.707705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.9.133.130:59423)
0807 06:11:52.708053 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:52.708066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:52.708198 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:52.708456 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:52.708459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:52.708851 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:52.709213 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.709220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.710140 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.710144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.710735 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.710743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.711309 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.711312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.711410 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.712029 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:52.712049 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:52.712438 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:52.714458 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.714461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.714464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.714691 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.715044 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.715047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.715051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.715141 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.715595 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:52.715598 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:52.715715 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 06:11:52.715832 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:11:52.716022 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":16,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:11:52.744771 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45551
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:11:52.884964 12661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:11:52.885289 12661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:11:52.885380 12661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:11:52.889700 12661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:11:52.889786 12661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:11:52.889830 12661 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 06:11:52.889846 12661 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:11:52.889883 12661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:11:52.896039 12661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:11:52.897428 12661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:11:52.898712 12661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:11:52.910905 12668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.910905 12672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:11:52.911093 12674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:11:52.911504 12661 server_base.cc:1034] running on GCE node
I20260807 06:11:52.912003 12661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:11:52.912799 12661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:11:52.914021 12661 hybrid_clock.cc:648] HybridClock initialized: now 1786083112914003 us; error 27 us; skew 500 ppm
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:11:52.917407 12661 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:11:52.918509 12661 webserver.cc:460] Webserver started at http://127.9.133.131:34171/ using document root <none> and password file <none>
I20260807 06:11:52.919284 12661 fs_manager.cc:359] Metadata directory not provided
I20260807 06:11:52.919396 12661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:11:52.919679 12661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:11:52.922196 12661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "5c4c26d56f1cc03a95b7343cd06dd479"
server_key_iv: "2c671a27cab074085c23a3a81dfda580"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.922932 12661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "5c4c26d56f1cc03a95b7343cd06dd479"
server_key_iv: "2c671a27cab074085c23a3a81dfda580"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.927901 12661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 06:11:52.930874 12682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.932394 12661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:11:52.932549 12661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
format_stamp: "Formatted at 2026-08-07 06:11:52 on dist-test-slave-h3ft"
server_key: "5c4c26d56f1cc03a95b7343cd06dd479"
server_key_iv: "2c671a27cab074085c23a3a81dfda580"
server_key_version: "encryptionkey@0"
I20260807 06:11:52.932694 12661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:11:52.946835 12661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:11:52.950762 12661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:11:52.951083 12661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:11:52.952996 12661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:11:52.953063 12661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:11:52.953110 12661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:11:52.953161 12661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:52.965868 12661 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:33413
I20260807 06:11:52.966033 12836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:33413 every 8 connection(s)
I20260807 06:11:52.967181 12661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:11:52.969991 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 12661
I20260807 06:11:52.970136 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:11:52.970568 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.953956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:48861 (local address 127.9.133.190:45551)
0807 06:11:52.954167 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.954175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.955216 (+ 1041us) server_negotiation.cc:407] Connection header received
0807 06:11:52.956817 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.956824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.956914 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.956993 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.959273 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.960093 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.961374 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.961630 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.961758 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.965699 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.965701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.965704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.965956 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.967904 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.968593 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.968597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.968599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.968778 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.969163 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.969166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.969168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.969440 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.969560 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.970269 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.970400 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 06:11:52.970570 9750 external_mini_cluster.cc:1428] Setting key 76660cff4536ea10bf9d1e16fa47fe53
I20260807 06:11:52.971861 12692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.954393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.9.133.131:48861)
0807 06:11:52.954968 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:52.955078 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:52.956484 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:52.957560 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:52.957570 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:52.958120 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:52.959069 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.959099 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.960241 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.960245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.961231 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.961241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.961825 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.961829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.962035 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.963536 (+ 1501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:52.963560 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:52.965452 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:52.968073 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.968078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.968094 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.968472 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.968868 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.968872 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.968874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.969051 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.969555 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:52.969562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:52.969951 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 06:11:52.970181 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 06:11:52.970574 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":127,"threads_started":1}
I20260807 06:11:52.973399 12837 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45551
I20260807 06:11:52.973559 12837 heartbeater.cc:464] Registering TS with master...
I20260807 06:11:52.974087 12837 heartbeater.cc:511] Master 127.9.133.190:45551 requested a full tablet report, sending...
I20260807 06:11:52.975443 12178 ts_manager.cc:194] Registered new tserver with Master: 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:52.976181 12178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:48861
I20260807 06:11:52.983115 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.975983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.9.133.131:32877)
0807 06:11:52.976259 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:52.976275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:52.976394 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:52.976627 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:52.976630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:52.976876 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:52.976990 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.976996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.978013 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.978015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.978603 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:52.978612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.979218 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.979221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:52.979302 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.979896 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:52.979908 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:52.980262 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:52.981774 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.981776 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.981778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.981972 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.982220 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:52.982222 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:52.982224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:52.982300 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:52.982635 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:52.982636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:52.982711 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 06:11:52.982790 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:11:52.982901 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
I20260807 06:11:52.983170 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:52.976045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:32877 (local address 127.9.133.190:45551)
0807 06:11:52.976166 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:52.976170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:52.976362 (+ 192us) server_negotiation.cc:407] Connection header received
0807 06:11:52.976465 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:52.976468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:52.976522 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:52.976598 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:52.977107 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.977874 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.978732 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:52.979083 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:52.979194 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:52.980391 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:52.980394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:52.980396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:52.980416 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:52.981679 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.982061 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.982064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.982066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.982140 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:52.982384 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:52.982387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:52.982388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:52.982555 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:52.982634 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:52.982852 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 06:11:52.982991 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:11:52.985100 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:11:52 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:11:53 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:53.017673 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.008609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.9.133.190:45551)
0807 06:11:53.008773 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.008777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.008792 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:11:53.008845 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.008847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.008895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.009008 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:53.009793 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.010394 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.011325 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.011556 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.011648 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.014494 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:53.014498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:53.014500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:53.014528 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:53.016095 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:53.016497 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:53.016500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:53.016501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:53.016576 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:53.016809 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:53.016811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:53.016812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:53.017033 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:53.017150 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.017346 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.017452 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:11:53.021303 12178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:11:53.025355 12178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:53.043113 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.036816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43583 (local address 127.0.0.1:54880)
0807 06:11:53.037307 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.037363 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.037574 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.038349 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.038356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.038394 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.038622 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.038652 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.040173 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.040177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.041400 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.041409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.042482 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.042486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.042595 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.042638 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.042762 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.042901 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":106,"threads_started":1}
I20260807 06:11:53.043154 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.037021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54880 (local address 127.9.133.130:43583)
0807 06:11:53.037725 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.037741 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.037773 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:11:53.037875 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.037884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.038153 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.038348 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.038804 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.040031 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.041530 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.042349 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.042466 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.042665 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.042811 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.042921 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":150,"threads_started":1}
I20260807 06:11:53.043964 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.037884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.9.133.131:33413)
0807 06:11:53.038757 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.038771 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.038798 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:11:53.038880 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.038888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.039098 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.039257 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.039446 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.040457 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.041780 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.042566 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.042662 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.042837 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.043685 (+ 848us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.043788 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":155,"threads_started":1}
I20260807 06:11:53.044232 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.038054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39095 (local address 127.0.0.1:40006)
0807 06:11:53.038354 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.038366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.038444 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.039395 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.039398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.039416 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.039541 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.039547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.041056 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.041059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.042029 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.042037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.043801 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.043804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.043902 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.043924 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.044011 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.044078 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
I20260807 06:11:53.044447 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.038448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40006 (local address 127.9.133.129:39095)
0807 06:11:53.038834 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.038841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.038868 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:11:53.038965 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.038970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.039240 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.039689 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.039836 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.040923 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.042151 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.043666 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.043791 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.043949 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.044157 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.044281 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
I20260807 06:11:53.044711 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.037666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:33413 (local address 127.0.0.1:44798)
0807 06:11:53.037958 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.037972 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.038065 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.039234 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.039237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.039256 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.039349 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.039355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.040579 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.040584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.041648 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.041657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.042904 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.042907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.043048 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.043072 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.043146 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.044079 (+ 933us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":132,"threads_started":1}
I20260807 06:11:53.047195 12551 tablet_service.cc:1467] Processing CreateTablet for tablet e944b65e9c584fa898ed978cd5254e52 (DEFAULT_TABLE table=test-table [id=31e34f081aff4913a7f8560ac6a1ef77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:53.047482 12361 tablet_service.cc:1467] Processing CreateTablet for tablet e944b65e9c584fa898ed978cd5254e52 (DEFAULT_TABLE table=test-table [id=31e34f081aff4913a7f8560ac6a1ef77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:53.047482 12751 tablet_service.cc:1467] Processing CreateTablet for tablet e944b65e9c584fa898ed978cd5254e52 (DEFAULT_TABLE table=test-table [id=31e34f081aff4913a7f8560ac6a1ef77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:53.048511 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e944b65e9c584fa898ed978cd5254e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:53.048514 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e944b65e9c584fa898ed978cd5254e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:53.048516 12551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e944b65e9c584fa898ed978cd5254e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:53.056260 12869 tablet_bootstrap.cc:492] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Bootstrap starting.
I20260807 06:11:53.057691 12872 tablet_bootstrap.cc:492] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Bootstrap starting.
I20260807 06:11:53.057690 12871 tablet_bootstrap.cc:492] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Bootstrap starting.
I20260807 06:11:53.060115 12869 tablet_bootstrap.cc:654] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:53.060127 12872 tablet_bootstrap.cc:654] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:53.060125 12871 tablet_bootstrap.cc:654] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:53.061028 12869 log.cc:824] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:53.061066 12872 log.cc:824] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:53.061066 12871 log.cc:824] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Log is configured to *not* fsync() on all Append() calls
I20260807 06:11:53.063457 12871 tablet_bootstrap.cc:492] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: No bootstrap required, opened a new log
I20260807 06:11:53.063431 12869 tablet_bootstrap.cc:492] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: No bootstrap required, opened a new log
I20260807 06:11:53.063724 12869 ts_tablet_manager.cc:1397] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 06:11:53.063726 12871 ts_tablet_manager.cc:1397] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 06:11:53.063887 12872 tablet_bootstrap.cc:492] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: No bootstrap required, opened a new log
I20260807 06:11:53.064093 12872 ts_tablet_manager.cc:1397] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:11:53.067543 12869 raft_consensus.cc:348] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.068259 12869 raft_consensus.cc:374] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:53.068322 12869 raft_consensus.cc:729] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Initialized, Role: FOLLOWER
I20260807 06:11:53.068210 12872 raft_consensus.cc:348] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.068210 12871 raft_consensus.cc:348] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.068487 12872 raft_consensus.cc:374] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:53.068500 12871 raft_consensus.cc:374] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:53.068549 12871 raft_consensus.cc:729] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e9ba3829dd42f4bf21f71f0f6d2167, State: Initialized, Role: FOLLOWER
I20260807 06:11:53.068560 12872 raft_consensus.cc:729] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Initialized, Role: FOLLOWER
I20260807 06:11:53.069111 12872 consensus_queue.cc:260] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.069085 12871 consensus_queue.cc:260] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.070361 12872 ts_tablet_manager.cc:1428] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 06:11:53.070361 12871 ts_tablet_manager.cc:1428] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 06:11:53.071229 12837 heartbeater.cc:503] Master 127.9.133.190:45551 was elected leader, sending a full tablet report...
I20260807 06:11:53.072422 12651 heartbeater.cc:503] Master 127.9.133.190:45551 was elected leader, sending a full tablet report...
I20260807 06:11:53.072770 12869 consensus_queue.cc:260] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.073683 12452 heartbeater.cc:503] Master 127.9.133.190:45551 was elected leader, sending a full tablet report...
I20260807 06:11:53.074057 12869 ts_tablet_manager.cc:1428] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260807 06:11:53.184950 12453 tablet.cc:2369] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:11:53.202020 12652 tablet.cc:2369] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:11:53.218472 12838 tablet.cc:2369] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:11:53.350013 12884 raft_consensus.cc:482] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:53.350381 12884 raft_consensus.cc:504] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.352013 12884 leader_election.cc:288] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:53.357658 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.352530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:44705 (local address 127.9.133.130:43583)
0807 06:11:53.352718 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.352722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.353053 (+ 331us) server_negotiation.cc:407] Connection header received
0807 06:11:53.353236 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.353242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.353318 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.353452 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.353792 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.354675 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.356148 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.356888 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.357064 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.357165 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.357443 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.357543 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 06:11:53.357659 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.352416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43583 (local address 127.9.133.129:44705)
0807 06:11:53.352867 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.352887 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.353049 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.353485 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.353487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.353508 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.353645 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.353653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.354806 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.354810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.355996 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.356006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.357081 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.357086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.357201 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.357234 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.357331 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.357441 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":219,"threads_started":1}
I20260807 06:11:53.358453 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.353288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:51929 (local address 127.9.133.131:33413)
0807 06:11:53.353455 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.353461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.353790 (+ 329us) server_negotiation.cc:407] Connection header received
0807 06:11:53.354051 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.354054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.354105 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.354182 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.354573 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.355450 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.357124 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.357773 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.357978 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.358050 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.358253 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.358341 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:11:53.358474 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.353069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:33413 (local address 127.9.133.129:51929)
0807 06:11:53.353631 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.353646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.353812 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.354233 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.354237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.354260 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.354438 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.354446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.355598 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.355601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.356973 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.356987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.357990 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.357994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.358095 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.358122 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.358209 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.358304 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260807 06:11:53.358685 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e944b65e9c584fa898ed978cd5254e52" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "267ff4b474c748e797e450485f1dd0e7" is_pre_election: true
I20260807 06:11:53.359192 12577 raft_consensus.cc:2454] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 0.
I20260807 06:11:53.359182 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e944b65e9c584fa898ed978cd5254e52" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" is_pre_election: true
I20260807 06:11:53.359515 12776 raft_consensus.cc:2454] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 0.
I20260807 06:11:53.359828 12299 leader_election.cc:302] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:53.360185 12884 raft_consensus.cc:2781] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:53.360291 12884 raft_consensus.cc:482] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:53.360340 12884 raft_consensus.cc:3037] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:53.361625 12884 raft_consensus.cc:504] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.362017 12884 leader_election.cc:288] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 election: Requested vote from peers 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:53.362468 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e944b65e9c584fa898ed978cd5254e52" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "267ff4b474c748e797e450485f1dd0e7"
I20260807 06:11:53.362488 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e944b65e9c584fa898ed978cd5254e52" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
I20260807 06:11:53.362597 12577 raft_consensus.cc:3037] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:53.362661 12776 raft_consensus.cc:3037] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:53.363997 12577 raft_consensus.cc:2454] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 1.
I20260807 06:11:53.364435 12299 leader_election.cc:302] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:53.364662 12884 raft_consensus.cc:2781] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:53.364925 12884 raft_consensus.cc:686] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 LEADER]: Becoming Leader. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Running, Role: LEADER
I20260807 06:11:53.365023 12776 raft_consensus.cc:2454] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 1.
I20260807 06:11:53.365415 12884 consensus_queue.cc:237] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } }
I20260807 06:11:53.369488 12177 catalog_manager.cc:5495] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "2844b4bcc165412dbf726ec23fa1fc55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } health_report { overall_health: HEALTHY } } }
Aug 07 06:11:53 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:53.445957 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.429999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.9.133.190:45551)
0807 06:11:53.430199 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.430205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.430425 (+ 220us) server_negotiation.cc:407] Connection header received
0807 06:11:53.430569 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.430572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.430619 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.430692 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:53.431576 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.432172 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.433075 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.433359 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.433504 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.442243 (+ 8739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:53.442247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:53.442250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:53.442278 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:53.443944 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:53.444518 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:53.444521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:53.444523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:53.444604 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:53.444921 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:53.444923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:53.444924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:53.445162 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:53.445243 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.445626 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.445833 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":74}
I20260807 06:11:53.445959 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.429892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45551 (local address 127.0.0.1:51934)
0807 06:11:53.430308 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.430324 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.430475 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.430735 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.430737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.431296 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:11:53.431427 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.431434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.432286 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.432292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.432949 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.432958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.433485 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.433488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.433564 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.440246 (+ 6682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:11:53.440261 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:11:53.442053 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:11:53.444095 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:53.444108 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:53.444115 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:53.444421 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:53.444712 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:11:53.444715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:11:53.444717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:11:53.444820 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:11:53.445305 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:11:53.445313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:11:53.445454 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.445569 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.445744 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":170,"threads_started":1}
I20260807 06:11:53.451280 12177 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:51934:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:11:53.452340 12177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:53.458815 12361 tablet_service.cc:1467] Processing CreateTablet for tablet 800c82f210e54de6bfe28004fa1de0bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89ffda62fc7242cdb9fb5ab0a3183ad4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:11:53.459316 12751 tablet_service.cc:1467] Processing CreateTablet for tablet 800c82f210e54de6bfe28004fa1de0bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89ffda62fc7242cdb9fb5ab0a3183ad4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:11:53.459270 12551 tablet_service.cc:1467] Processing CreateTablet for tablet 800c82f210e54de6bfe28004fa1de0bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89ffda62fc7242cdb9fb5ab0a3183ad4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:11:53.460041 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 800c82f210e54de6bfe28004fa1de0bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:53.460042 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 800c82f210e54de6bfe28004fa1de0bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:53.460183 12551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 800c82f210e54de6bfe28004fa1de0bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:53.463958 12869 tablet_bootstrap.cc:492] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: Bootstrap starting.
I20260807 06:11:53.464005 12872 tablet_bootstrap.cc:492] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: Bootstrap starting.
I20260807 06:11:53.464298 12871 tablet_bootstrap.cc:492] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: Bootstrap starting.
I20260807 06:11:53.465313 12872 tablet_bootstrap.cc:654] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:53.465406 12871 tablet_bootstrap.cc:654] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:53.465402 12869 tablet_bootstrap.cc:654] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:53.467044 12871 tablet_bootstrap.cc:492] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: No bootstrap required, opened a new log
I20260807 06:11:53.467075 12872 tablet_bootstrap.cc:492] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: No bootstrap required, opened a new log
I20260807 06:11:53.467163 12871 ts_tablet_manager.cc:1397] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:11:53.467186 12872 ts_tablet_manager.cc:1397] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:11:53.467388 12869 tablet_bootstrap.cc:492] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: No bootstrap required, opened a new log
I20260807 06:11:53.467512 12869 ts_tablet_manager.cc:1397] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:11:53.467767 12871 raft_consensus.cc:348] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.467922 12871 raft_consensus.cc:374] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:53.467958 12871 raft_consensus.cc:729] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e9ba3829dd42f4bf21f71f0f6d2167, State: Initialized, Role: FOLLOWER
I20260807 06:11:53.467882 12872 raft_consensus.cc:348] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.468034 12872 raft_consensus.cc:374] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:53.468076 12872 raft_consensus.cc:729] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Initialized, Role: FOLLOWER
I20260807 06:11:53.468102 12871 consensus_queue.cc:260] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.468271 12872 consensus_queue.cc:260] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.468443 12871 ts_tablet_manager.cc:1428] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 06:11:53.468490 12883 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.468654 12872 ts_tablet_manager.cc:1428] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:11:53.468717 12883 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.468791 12882 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.468170 12869 raft_consensus.cc:348] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.468966 12882 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.468978 12869 raft_consensus.cc:374] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:53.469048 12869 raft_consensus.cc:729] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Initialized, Role: FOLLOWER
I20260807 06:11:53.469230 12869 consensus_queue.cc:260] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.469542 12884 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.469712 12884 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.469612 12869 ts_tablet_manager.cc:1428] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 06:11:53.838178 12882 raft_consensus.cc:482] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:53.838428 12882 raft_consensus.cc:504] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.839998 12882 leader_election.cc:288] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:53.845074 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.840465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39095 (local address 127.9.133.130:49053)
0807 06:11:53.840788 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.840803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.840980 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.841404 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.841407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.841430 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.841611 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.841620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.842724 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.842729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.843767 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.843777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.844610 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.844615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.844718 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.844737 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.844839 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.844908 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":4,"thread_start_us":122,"threads_started":1}
I20260807 06:11:53.845059 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.841165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:47105 (local address 127.9.133.131:33413)
0807 06:11:53.841318 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.841322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.841495 (+ 173us) server_negotiation.cc:407] Connection header received
0807 06:11:53.841659 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.841665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.841735 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.841805 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.842128 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.842676 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.843971 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.844402 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.844511 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.844607 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.844806 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.844899 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 06:11:53.845099 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.840989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:33413 (local address 127.9.133.130:47105)
0807 06:11:53.841379 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.841406 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.841530 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.841850 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.841853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.841875 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:11:53.842013 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.842021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.842824 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.842827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.843774 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.843784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.844538 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.844545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.844655 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.844682 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.844770 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.844879 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":10,"spinlock_wait_cycles":1920,"thread_start_us":96,"threads_started":1}
I20260807 06:11:53.845402 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.840530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:49053 (local address 127.9.133.129:39095)
0807 06:11:53.840861 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.840870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.840906 (+ 36us) server_negotiation.cc:407] Connection header received
0807 06:11:53.841154 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.841158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.841240 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.841372 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:11:53.841761 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.842591 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.843928 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.844436 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.844614 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.844678 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.844890 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.845001 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":86,"threads_started":1}
I20260807 06:11:53.845620 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800c82f210e54de6bfe28004fa1de0bd" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" is_pre_election: true
I20260807 06:11:53.845829 12776 raft_consensus.cc:2454] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 0.
I20260807 06:11:53.846210 12389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800c82f210e54de6bfe28004fa1de0bd" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2844b4bcc165412dbf726ec23fa1fc55" is_pre_election: true
I20260807 06:11:53.846258 12489 leader_election.cc:302] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 48e9ba3829dd42f4bf21f71f0f6d2167; no voters:
I20260807 06:11:53.846526 12389 raft_consensus.cc:2454] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 0.
I20260807 06:11:53.846577 12882 raft_consensus.cc:2781] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:53.846661 12882 raft_consensus.cc:482] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:53.846709 12882 raft_consensus.cc:3037] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:53.848362 12882 raft_consensus.cc:504] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.848759 12882 leader_election.cc:288] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 election: Requested vote from peers 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:53.849220 12389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800c82f210e54de6bfe28004fa1de0bd" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2844b4bcc165412dbf726ec23fa1fc55"
I20260807 06:11:53.849233 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800c82f210e54de6bfe28004fa1de0bd" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
I20260807 06:11:53.849340 12776 raft_consensus.cc:3037] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:53.849380 12389 raft_consensus.cc:3037] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:53.850414 12776 raft_consensus.cc:2454] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 1.
I20260807 06:11:53.850695 12389 raft_consensus.cc:2454] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 1.
I20260807 06:11:53.850735 12489 leader_election.cc:302] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 48e9ba3829dd42f4bf21f71f0f6d2167; no voters:
I20260807 06:11:53.850924 12882 raft_consensus.cc:2781] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:53.851174 12882 raft_consensus.cc:686] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [term 1 LEADER]: Becoming Leader. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Running, Role: LEADER
I20260807 06:11:53.851635 12882 consensus_queue.cc:237] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:53.854367 12904 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: TxnStatusTablet state changed. Reason: New leader 267ff4b474c748e797e450485f1dd0e7. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.854504 12904 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:11:53.854851 12908 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:11:53.855132 12908 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:11:53.855705 12178 catalog_manager.cc:5495] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130). New cstate: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
I20260807 06:11:53.910380 12884 consensus_queue.cc:1035] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:53.914830 12884 consensus_queue.cc:1035] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:53.932782 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.927856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.9.133.130:43583)
0807 06:11:53.928222 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.928231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.928247 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:11:53.928329 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.928333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.928404 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.928539 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:53.928797 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.929715 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.930732 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.931137 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.931245 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.931576 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:53.932238 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:53.932362 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.932492 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.932563 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":168,"threads_started":1}
I20260807 06:11:53.932765 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.927735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43583 (local address 127.0.0.1:54888)
0807 06:11:53.928019 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.928043 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.928174 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.928555 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.928558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.928565 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:53.928687 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.928692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.929900 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.929903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.930594 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.930605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.931259 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.931262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.931340 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.931396 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:53.932362 (+ 966us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:53.932380 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.932458 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.932556 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 06:11:53.939495 12776 raft_consensus.cc:1261] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Refusing update from remote peer 267ff4b474c748e797e450485f1dd0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:53.939805 12389 raft_consensus.cc:1261] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Refusing update from remote peer 267ff4b474c748e797e450485f1dd0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:53.940562 12882 consensus_queue.cc:1035] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:53.941133 12882 consensus_queue.cc:1035] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:53.945436 12920 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: TxnStatusTablet state changed. Reason: New leader 267ff4b474c748e797e450485f1dd0e7. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.945608 12920 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.946049 12884 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: TxnStatusTablet state changed. Reason: New leader 267ff4b474c748e797e450485f1dd0e7. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.946193 12884 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.949450 12882 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.949584 12882 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:11:53.949942 12919 mvcc.cc:205] Tried to move back new op lower bound from 7315796434687987712 to 7315796434338959360. Current Snapshot: MvccSnapshot[applied={T|T < 7315796434687987712}]
I20260807 06:11:53.950073 12920 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.950199 12920 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 48e9ba3829dd42f4bf21f71f0f6d2167: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.953536 12916 mvcc.cc:205] Tried to move back new op lower bound from 7315796434687987712 to 7315796434338959360. Current Snapshot: MvccSnapshot[applied={T|T < 7315796434687987712 or (T in {7315796434687987712})}]
I20260807 06:11:53.955147 12882 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.955297 12882 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 267ff4b474c748e797e450485f1dd0e7: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:11:53.957083 12884 tablet_replica.cc:439] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } } }
I20260807 06:11:53.957228 12884 tablet_replica.cc:442] T 800c82f210e54de6bfe28004fa1de0bd P 2844b4bcc165412dbf726ec23fa1fc55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:11:53.957387 12915 mvcc.cc:205] Tried to move back new op lower bound from 7315796434687987712 to 7315796434338959360. Current Snapshot: MvccSnapshot[applied={T|T < 7315796434687987712}]
I20260807 06:11:53.964258 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.959265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.9.133.129:39095)
0807 06:11:53.959380 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.959384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.959399 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:11:53.959537 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.959540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.959611 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.959725 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:53.960037 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.960858 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.962249 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.962518 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.962617 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.962970 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:53.963630 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:53.963737 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.963867 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.964028 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 06:11:53.980813 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.975490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:40887 (local address 127.9.133.130:43583)
0807 06:11:53.975648 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:53.975653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:53.975965 (+ 312us) server_negotiation.cc:407] Connection header received
0807 06:11:53.976139 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:53.976143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:53.976230 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:53.976315 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:53.976814 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.977774 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.979233 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.979638 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.979717 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.980216 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:53.980322 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:53.980380 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:53.980550 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 06:11:53.980624 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:11:53.981305 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:53.975371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43583 (local address 127.9.133.129:40887)
0807 06:11:53.975855 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:53.975872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:53.976007 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:53.976461 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:53.976464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:53.976474 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:53.976645 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.976656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.977942 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.977945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.979086 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:53.979098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:53.979846 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:53.979849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:53.979962 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:53.980022 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:53.980446 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:53.980452 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:11:53.981019 (+ 567us) client_negotiation.cc:240] Negotiation successful
0807 06:11:53.981130 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":156,"threads_started":1}
I20260807 06:11:54.017508 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:54.011802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:48535 (local address 127.9.133.129:39095)
0807 06:11:54.011992 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:54.011998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:54.012175 (+ 177us) server_negotiation.cc:407] Connection header received
0807 06:11:54.012311 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:54.012318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:54.012423 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:54.012526 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:54.012888 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:54.013739 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:54.015876 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:54.016273 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:54.016379 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:54.016864 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:54.016996 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:54.017103 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:54.017208 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 06:11:54.017283 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:11:54.017515 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:54.011719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39095 (local address 127.9.133.130:48535)
0807 06:11:54.012063 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:54.012080 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:54.012209 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:54.012541 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:54.012545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:54.012555 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:54.012733 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:54.012741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:54.013866 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:54.013869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:54.015738 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:54.015752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:54.016502 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:54.016506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:54.016611 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:54.016677 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:54.017112 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:54.017122 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 06:11:54.017195 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:11:54.017301 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":107,"threads_started":1}
I20260807 06:11:54.046249 12176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:54.046514 12176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:54.049685 12176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Sending DeleteTablet for 3 replicas of tablet e944b65e9c584fa898ed978cd5254e52
I20260807 06:11:54.050612 12551 tablet_service.cc:1514] Processing DeleteTablet for tablet e944b65e9c584fa898ed978cd5254e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:54 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:54880
I20260807 06:11:54.050719 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet e944b65e9c584fa898ed978cd5254e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:54 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44798
I20260807 06:11:54.050908 12361 tablet_service.cc:1514] Processing DeleteTablet for tablet e944b65e9c584fa898ed978cd5254e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:54 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:40006
I20260807 06:11:54.051625 12944 tablet_replica.cc:330] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: stopping tablet replica
I20260807 06:11:54.051668 12947 tablet_replica.cc:330] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: stopping tablet replica
I20260807 06:11:54.051754 12176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 06:11:54.052115 12947 raft_consensus.cc:2229] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 LEADER]: Raft consensus shutting down.
W20260807 06:11:54.052307 12176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:54.052346 12944 raft_consensus.cc:2229] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:54.052628 12947 raft_consensus.cc:2258] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:54.052878 12944 raft_consensus.cc:2258] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:54.053354 12945 tablet_replica.cc:330] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: stopping tablet replica
I20260807 06:11:54.053731 12945 raft_consensus.cc:2229] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:54.054308 12947 ts_tablet_manager.cc:1905] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:54.054564 12945 raft_consensus.cc:2258] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:54.055064 12944 ts_tablet_manager.cc:1905] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:54.056012 12945 ts_tablet_manager.cc:1905] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:54.058676 12551 tablet_service.cc:1467] Processing CreateTablet for tablet 0993a12f180e4f198e4cfdac684c4b26 (DEFAULT_TABLE table=test-table [id=17e3762ad89543cd9b4f6b518a623eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:54.058848 12944 ts_tablet_manager.cc:1918] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:54.058871 12361 tablet_service.cc:1467] Processing CreateTablet for tablet 0993a12f180e4f198e4cfdac684c4b26 (DEFAULT_TABLE table=test-table [id=17e3762ad89543cd9b4f6b518a623eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:54.058964 12944 log.cc:1196] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/e944b65e9c584fa898ed978cd5254e52
I20260807 06:11:54.059032 12551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0993a12f180e4f198e4cfdac684c4b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:54.059329 12944 ts_tablet_manager.cc:1939] T e944b65e9c584fa898ed978cd5254e52 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting consensus metadata
I20260807 06:11:54.059264 12751 tablet_service.cc:1467] Processing CreateTablet for tablet 0993a12f180e4f198e4cfdac684c4b26 (DEFAULT_TABLE table=test-table [id=17e3762ad89543cd9b4f6b518a623eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:54.059579 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0993a12f180e4f198e4cfdac684c4b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:54.059585 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0993a12f180e4f198e4cfdac684c4b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:54.060110 12945 ts_tablet_manager.cc:1918] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:54.060250 12945 log.cc:1196] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/e944b65e9c584fa898ed978cd5254e52
I20260807 06:11:54.060617 12945 ts_tablet_manager.cc:1939] T e944b65e9c584fa898ed978cd5254e52 P 267ff4b474c748e797e450485f1dd0e7: Deleting consensus metadata
I20260807 06:11:54.062162 12156 catalog_manager.cc:4841] TS 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583): tablet e944b65e9c584fa898ed978cd5254e52 (table test-table [id=31e34f081aff4913a7f8560ac6a1ef77]) successfully deleted
I20260807 06:11:54.062731 12948 tablet_bootstrap.cc:492] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Bootstrap starting.
I20260807 06:11:54.063288 12952 tablet_bootstrap.cc:492] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Bootstrap starting.
I20260807 06:11:54.063546 12156 catalog_manager.cc:4841] TS 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413): tablet e944b65e9c584fa898ed978cd5254e52 (table test-table [id=31e34f081aff4913a7f8560ac6a1ef77]) successfully deleted
I20260807 06:11:54.063928 12954 tablet_bootstrap.cc:492] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Bootstrap starting.
I20260807 06:11:54.065079 12954 tablet_bootstrap.cc:654] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:54.065413 12948 tablet_bootstrap.cc:654] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:54.066375 12952 tablet_bootstrap.cc:654] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:54.066694 12954 tablet_bootstrap.cc:492] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: No bootstrap required, opened a new log
I20260807 06:11:54.066797 12954 ts_tablet_manager.cc:1397] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:11:54.067421 12954 raft_consensus.cc:348] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.067581 12954 raft_consensus.cc:374] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:54.067623 12954 raft_consensus.cc:729] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Initialized, Role: FOLLOWER
I20260807 06:11:54.067772 12954 consensus_queue.cc:260] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.068081 12954 ts_tablet_manager.cc:1428] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 06:11:54.068284 12948 tablet_bootstrap.cc:492] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: No bootstrap required, opened a new log
I20260807 06:11:54.068426 12948 ts_tablet_manager.cc:1397] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 06:11:54.068928 12948 raft_consensus.cc:348] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.069126 12948 raft_consensus.cc:374] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:54.069212 12948 raft_consensus.cc:729] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Initialized, Role: FOLLOWER
I20260807 06:11:54.069370 12948 consensus_queue.cc:260] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.069778 12948 ts_tablet_manager.cc:1428] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:11:54.070633 12952 tablet_bootstrap.cc:492] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: No bootstrap required, opened a new log
I20260807 06:11:54.070768 12952 ts_tablet_manager.cc:1397] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260807 06:11:54.071328 12952 raft_consensus.cc:348] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.071468 12952 raft_consensus.cc:374] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:54.071507 12952 raft_consensus.cc:729] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e9ba3829dd42f4bf21f71f0f6d2167, State: Initialized, Role: FOLLOWER
I20260807 06:11:54.071676 12952 consensus_queue.cc:260] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.072021 12952 ts_tablet_manager.cc:1428] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260807 06:11:54.071862 12947 ts_tablet_manager.cc:1918] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:54.074088 12947 log.cc:1196] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/e944b65e9c584fa898ed978cd5254e52
I20260807 06:11:54.075368 12947 ts_tablet_manager.cc:1939] T e944b65e9c584fa898ed978cd5254e52 P 2844b4bcc165412dbf726ec23fa1fc55: Deleting consensus metadata
I20260807 06:11:54.076833 12156 catalog_manager.cc:4841] TS 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095): tablet e944b65e9c584fa898ed978cd5254e52 (table test-table [id=31e34f081aff4913a7f8560ac6a1ef77]) successfully deleted
I20260807 06:11:54.159421 12912 raft_consensus.cc:482] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:54.159610 12912 raft_consensus.cc:504] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.160269 12912 leader_election.cc:288] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:54.160933 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0993a12f180e4f198e4cfdac684c4b26" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "267ff4b474c748e797e450485f1dd0e7" is_pre_election: true
I20260807 06:11:54.160976 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0993a12f180e4f198e4cfdac684c4b26" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" is_pre_election: true
I20260807 06:11:54.161149 12776 raft_consensus.cc:2454] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 0.
I20260807 06:11:54.161149 12577 raft_consensus.cc:2454] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 0.
I20260807 06:11:54.161553 12299 leader_election.cc:302] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:54.161803 12912 raft_consensus.cc:2781] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:54.161887 12912 raft_consensus.cc:482] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:54.161968 12912 raft_consensus.cc:3037] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:54.163539 12912 raft_consensus.cc:504] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.164103 12912 leader_election.cc:288] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 election: Requested vote from peers 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:54.164572 12577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0993a12f180e4f198e4cfdac684c4b26" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "267ff4b474c748e797e450485f1dd0e7"
I20260807 06:11:54.164592 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0993a12f180e4f198e4cfdac684c4b26" candidate_uuid: "2844b4bcc165412dbf726ec23fa1fc55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
I20260807 06:11:54.164737 12776 raft_consensus.cc:3037] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:54.164737 12577 raft_consensus.cc:3037] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:54.166137 12776 raft_consensus.cc:2454] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 1.
I20260807 06:11:54.166137 12577 raft_consensus.cc:2454] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2844b4bcc165412dbf726ec23fa1fc55 in term 1.
I20260807 06:11:54.166507 12299 leader_election.cc:302] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:54.166817 12912 raft_consensus.cc:2781] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:54.167037 12912 raft_consensus.cc:686] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 LEADER]: Becoming Leader. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Running, Role: LEADER
I20260807 06:11:54.167220 12912 consensus_queue.cc:237] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:54.169407 12178 catalog_manager.cc:5495] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "2844b4bcc165412dbf726ec23fa1fc55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
I20260807 06:11:54.211421 12776 raft_consensus.cc:1261] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Refusing update from remote peer 2844b4bcc165412dbf726ec23fa1fc55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:54.211467 12577 raft_consensus.cc:1261] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Refusing update from remote peer 2844b4bcc165412dbf726ec23fa1fc55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:54.212014 12884 consensus_queue.cc:1035] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:54.212273 12914 consensus_queue.cc:1035] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:11:54.232199 12915 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 06:11:55.793642 12176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:55.793860 12176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:55.796861 12176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Sending DeleteTablet for 3 replicas of tablet 0993a12f180e4f198e4cfdac684c4b26
I20260807 06:11:55.797559 12361 tablet_service.cc:1514] Processing DeleteTablet for tablet 0993a12f180e4f198e4cfdac684c4b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:55 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:40006
I20260807 06:11:55.797760 12551 tablet_service.cc:1514] Processing DeleteTablet for tablet 0993a12f180e4f198e4cfdac684c4b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:55 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:54880
I20260807 06:11:55.797768 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet 0993a12f180e4f198e4cfdac684c4b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:55 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44798
I20260807 06:11:55.798233 12998 tablet_replica.cc:330] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: stopping tablet replica
I20260807 06:11:55.798434 12998 raft_consensus.cc:2229] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:55.798599 12998 raft_consensus.cc:2258] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:55.798892 12997 tablet_replica.cc:330] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: stopping tablet replica
I20260807 06:11:55.799099 12997 raft_consensus.cc:2229] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:55.799260 12997 raft_consensus.cc:2258] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:55.799531 12998 ts_tablet_manager.cc:1905] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:55.800099 12997 ts_tablet_manager.cc:1905] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:55.803224 12998 ts_tablet_manager.cc:1918] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:11:55.803392 12998 log.cc:1196] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/0993a12f180e4f198e4cfdac684c4b26
I20260807 06:11:55.803722 12998 ts_tablet_manager.cc:1939] T 0993a12f180e4f198e4cfdac684c4b26 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting consensus metadata
I20260807 06:11:55.803711 12997 ts_tablet_manager.cc:1918] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:11:55.803790 12997 log.cc:1196] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/0993a12f180e4f198e4cfdac684c4b26
I20260807 06:11:55.804075 12997 ts_tablet_manager.cc:1939] T 0993a12f180e4f198e4cfdac684c4b26 P 267ff4b474c748e797e450485f1dd0e7: Deleting consensus metadata
I20260807 06:11:55.804064 12996 tablet_replica.cc:330] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: stopping tablet replica
I20260807 06:11:55.804268 12996 raft_consensus.cc:2229] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:11:55.804633 12996 raft_consensus.cc:2258] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:55.804782 12156 catalog_manager.cc:4841] TS 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413): tablet 0993a12f180e4f198e4cfdac684c4b26 (table test-table [id=17e3762ad89543cd9b4f6b518a623eff]) successfully deleted
I20260807 06:11:55.805056 12156 catalog_manager.cc:4841] TS 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583): tablet 0993a12f180e4f198e4cfdac684c4b26 (table test-table [id=17e3762ad89543cd9b4f6b518a623eff]) successfully deleted
I20260807 06:11:55.805742 12996 ts_tablet_manager.cc:1905] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:55.809659 12996 ts_tablet_manager.cc:1918] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:11:55.809767 12996 log.cc:1196] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/0993a12f180e4f198e4cfdac684c4b26
I20260807 06:11:55.810075 12996 ts_tablet_manager.cc:1939] T 0993a12f180e4f198e4cfdac684c4b26 P 2844b4bcc165412dbf726ec23fa1fc55: Deleting consensus metadata
I20260807 06:11:55.810938 12156 catalog_manager.cc:4841] TS 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095): tablet 0993a12f180e4f198e4cfdac684c4b26 (table test-table [id=17e3762ad89543cd9b4f6b518a623eff]) successfully deleted
Aug 07 06:11:55 dist-test-slave-h3ft krb5kdc[12115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:11:55 dist-test-slave-h3ft krb5kdc[12115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:11:55.836447 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:55.825882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.9.133.190:45551)
0807 06:11:55.826273 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:55.826284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:55.826305 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:11:55.826503 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:55.826507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:55.826566 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:55.826681 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:11:55.827558 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:55.828231 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:55.829166 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:55.829435 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:55.829572 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:55.832517 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:11:55.832521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:11:55.832524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:11:55.832555 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:11:55.834430 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:55.834917 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:55.834921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:55.834923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:55.835071 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:11:55.835395 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:11:55.835398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:11:55.835400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:11:55.835648 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:11:55.835809 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:55.836067 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 06:11:55.836225 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":142,"threads_started":1}
I20260807 06:11:55.839885 12176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:11:55.840515 12176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:55.846045 12361 tablet_service.cc:1467] Processing CreateTablet for tablet 807048d6ca6e4e5fb89d2a45704e557c (DEFAULT_TABLE table=test-table [id=9eabfccd33cb4c31ab6d5064379faab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:55.846236 12751 tablet_service.cc:1467] Processing CreateTablet for tablet 807048d6ca6e4e5fb89d2a45704e557c (DEFAULT_TABLE table=test-table [id=9eabfccd33cb4c31ab6d5064379faab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:55.846467 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807048d6ca6e4e5fb89d2a45704e557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:55.846611 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807048d6ca6e4e5fb89d2a45704e557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:55.849253 13008 tablet_bootstrap.cc:492] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Bootstrap starting.
I20260807 06:11:55.849506 13009 tablet_bootstrap.cc:492] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Bootstrap starting.
I20260807 06:11:55.849552 12551 tablet_service.cc:1467] Processing CreateTablet for tablet 807048d6ca6e4e5fb89d2a45704e557c (DEFAULT_TABLE table=test-table [id=9eabfccd33cb4c31ab6d5064379faab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:55.849937 12551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807048d6ca6e4e5fb89d2a45704e557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:55.850888 13009 tablet_bootstrap.cc:654] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:55.852663 13008 tablet_bootstrap.cc:654] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:55.853055 13017 tablet_bootstrap.cc:492] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Bootstrap starting.
I20260807 06:11:55.853766 13009 tablet_bootstrap.cc:492] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: No bootstrap required, opened a new log
I20260807 06:11:55.853878 13009 ts_tablet_manager.cc:1397] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 06:11:55.854104 13017 tablet_bootstrap.cc:654] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:55.854499 13009 raft_consensus.cc:348] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:55.854661 13009 raft_consensus.cc:374] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:55.854715 13009 raft_consensus.cc:729] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e9ba3829dd42f4bf21f71f0f6d2167, State: Initialized, Role: FOLLOWER
I20260807 06:11:55.854872 13009 consensus_queue.cc:260] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:55.855338 13009 ts_tablet_manager.cc:1428] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:11:55.855535 13017 tablet_bootstrap.cc:492] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: No bootstrap required, opened a new log
I20260807 06:11:55.855647 13017 ts_tablet_manager.cc:1397] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:11:55.855950 13008 tablet_bootstrap.cc:492] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: No bootstrap required, opened a new log
I20260807 06:11:55.856066 13008 ts_tablet_manager.cc:1397] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 06:11:55.856328 13017 raft_consensus.cc:348] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:55.856451 13017 raft_consensus.cc:374] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:55.856485 13017 raft_consensus.cc:729] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Initialized, Role: FOLLOWER
I20260807 06:11:55.856540 13008 raft_consensus.cc:348] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:55.856662 13008 raft_consensus.cc:374] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:55.856705 13008 raft_consensus.cc:729] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Initialized, Role: FOLLOWER
I20260807 06:11:55.856675 13017 consensus_queue.cc:260] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:55.856880 13008 consensus_queue.cc:260] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:55.857195 13008 ts_tablet_manager.cc:1428] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:11:55.857465 13017 ts_tablet_manager.cc:1428] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:11:56.232204 12882 raft_consensus.cc:482] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:56.232445 12882 raft_consensus.cc:504] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.233023 12882 leader_election.cc:288] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:56.233739 12389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807048d6ca6e4e5fb89d2a45704e557c" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2844b4bcc165412dbf726ec23fa1fc55" is_pre_election: true
I20260807 06:11:56.233801 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807048d6ca6e4e5fb89d2a45704e557c" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" is_pre_election: true
I20260807 06:11:56.233979 12389 raft_consensus.cc:2454] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 0.
I20260807 06:11:56.234019 12776 raft_consensus.cc:2454] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 0.
I20260807 06:11:56.234433 12489 leader_election.cc:302] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 48e9ba3829dd42f4bf21f71f0f6d2167; no voters:
I20260807 06:11:56.234670 12882 raft_consensus.cc:2781] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:56.234740 12882 raft_consensus.cc:482] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:56.234768 12882 raft_consensus.cc:3037] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:56.236099 12882 raft_consensus.cc:504] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.236488 12882 leader_election.cc:288] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 election: Requested vote from peers 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:56.237032 12389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807048d6ca6e4e5fb89d2a45704e557c" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2844b4bcc165412dbf726ec23fa1fc55"
I20260807 06:11:56.237118 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807048d6ca6e4e5fb89d2a45704e557c" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
I20260807 06:11:56.237190 12389 raft_consensus.cc:3037] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:56.237284 12776 raft_consensus.cc:3037] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:56.238644 12389 raft_consensus.cc:2454] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 1.
I20260807 06:11:56.238643 12776 raft_consensus.cc:2454] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 1.
I20260807 06:11:56.239081 12489 leader_election.cc:302] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:56.239261 12882 raft_consensus.cc:2781] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:56.239382 12882 raft_consensus.cc:686] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 1 LEADER]: Becoming Leader. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Running, Role: LEADER
I20260807 06:11:56.239598 12882 consensus_queue.cc:237] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.241684 12177 catalog_manager.cc:5495] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130). New cstate: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
I20260807 06:11:56.317344 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:56.313859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.9.133.130:43583)
0807 06:11:56.314134 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:56.314138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:56.314152 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:11:56.314204 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:56.314207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:56.314256 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:56.314334 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:56.314583 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:56.315333 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:56.316259 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:56.316484 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:56.316567 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:56.316803 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:56.316885 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:56.316946 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:56.317064 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 06:11:56.317115 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":137,"threads_started":1}
I20260807 06:11:56.327652 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:56.324479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:52937 (local address 127.9.133.130:43583)
0807 06:11:56.324595 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 06:11:56.324598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:11:56.324783 (+ 185us) server_negotiation.cc:407] Connection header received
0807 06:11:56.324884 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:11:56.324888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:11:56.324938 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:11:56.325009 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:11:56.325347 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:56.325853 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:56.326662 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:11:56.326866 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:11:56.326958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:56.327235 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:11:56.327302 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:11:56.327360 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:11:56.327457 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 06:11:56.327503 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:11:56.327657 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:56.324376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43583 (local address 127.9.133.130:52937)
0807 06:11:56.324695 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:56.324710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:11:56.324804 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:11:56.325066 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:11:56.325070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:11:56.325084 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:11:56.325254 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:56.325260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:56.325945 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:56.325947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:56.326560 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:11:56.326568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:11:56.326970 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:11:56.326972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:11:56.327107 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:11:56.327133 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:11:56.327377 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:11:56.327381 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:11:56.327431 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:11:56.327504 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":173,"threads_started":1}
I20260807 06:11:56.334158 12776 raft_consensus.cc:1261] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Refusing update from remote peer 267ff4b474c748e797e450485f1dd0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:56.334180 12389 raft_consensus.cc:1261] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Refusing update from remote peer 267ff4b474c748e797e450485f1dd0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:56.334573 13025 consensus_queue.cc:1035] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:56.334770 12882 consensus_queue.cc:1035] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:56.370553 12177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:56.370774 12177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:56.373323 12177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Sending DeleteTablet for 3 replicas of tablet 807048d6ca6e4e5fb89d2a45704e557c
I20260807 06:11:56.374040 12361 tablet_service.cc:1514] Processing DeleteTablet for tablet 807048d6ca6e4e5fb89d2a45704e557c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:56 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:40006
I20260807 06:11:56.374155 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet 807048d6ca6e4e5fb89d2a45704e557c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:56 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44798
I20260807 06:11:56.374627 12550 tablet_service.cc:1514] Processing DeleteTablet for tablet 807048d6ca6e4e5fb89d2a45704e557c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:56 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:54880
I20260807 06:11:56.374689 13042 tablet_replica.cc:330] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: stopping tablet replica
I20260807 06:11:56.374833 12177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 06:11:56.374969 13045 tablet_replica.cc:330] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: stopping tablet replica
I20260807 06:11:56.375133 13045 raft_consensus.cc:2229] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:11:56.374951 13042 raft_consensus.cc:2229] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:56.375375 13042 raft_consensus.cc:2258] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 06:11:56.375381 12177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:11:56.375516 13045 raft_consensus.cc:2258] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:56.376305 13042 ts_tablet_manager.cc:1905] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:56.376304 13045 ts_tablet_manager.cc:1905] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:56.379469 13045 ts_tablet_manager.cc:1918] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:56.379565 13045 log.cc:1196] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/807048d6ca6e4e5fb89d2a45704e557c
I20260807 06:11:56.379820 13042 ts_tablet_manager.cc:1918] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:56.379856 13045 ts_tablet_manager.cc:1939] T 807048d6ca6e4e5fb89d2a45704e557c P 267ff4b474c748e797e450485f1dd0e7: Deleting consensus metadata
I20260807 06:11:56.380056 13042 log.cc:1196] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/807048d6ca6e4e5fb89d2a45704e557c
I20260807 06:11:56.380414 13042 ts_tablet_manager.cc:1939] T 807048d6ca6e4e5fb89d2a45704e557c P 2844b4bcc165412dbf726ec23fa1fc55: Deleting consensus metadata
I20260807 06:11:56.381188 12361 tablet_service.cc:1467] Processing CreateTablet for tablet c1316bd74ab34ed1b1afff527d0b1ee1 (DEFAULT_TABLE table=test-table [id=bfe300fb8a24489b82207f7778017ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:56.381481 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1316bd74ab34ed1b1afff527d0b1ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:56.381597 13044 tablet_replica.cc:330] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: stopping tablet replica
I20260807 06:11:56.381765 13044 raft_consensus.cc:2229] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:56.382040 13044 raft_consensus.cc:2258] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:56.382700 12550 tablet_service.cc:1467] Processing CreateTablet for tablet c1316bd74ab34ed1b1afff527d0b1ee1 (DEFAULT_TABLE table=test-table [id=bfe300fb8a24489b82207f7778017ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:56.383034 12550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1316bd74ab34ed1b1afff527d0b1ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:56.384140 13047 tablet_bootstrap.cc:492] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55: Bootstrap starting.
I20260807 06:11:56.385136 13047 tablet_bootstrap.cc:654] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:56.385406 13048 tablet_bootstrap.cc:492] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: Bootstrap starting.
I20260807 06:11:56.385579 12156 catalog_manager.cc:4841] TS 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095): tablet 807048d6ca6e4e5fb89d2a45704e557c (table test-table [id=9eabfccd33cb4c31ab6d5064379faab0]) successfully deleted
I20260807 06:11:56.386400 13047 tablet_bootstrap.cc:492] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55: No bootstrap required, opened a new log
I20260807 06:11:56.386340 12751 tablet_service.cc:1467] Processing CreateTablet for tablet c1316bd74ab34ed1b1afff527d0b1ee1 (DEFAULT_TABLE table=test-table [id=bfe300fb8a24489b82207f7778017ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:11:56.386479 13047 ts_tablet_manager.cc:1397] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 06:11:56.386656 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1316bd74ab34ed1b1afff527d0b1ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:11:56.386879 13047 raft_consensus.cc:348] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.386992 13047 raft_consensus.cc:374] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:56.387075 13047 raft_consensus.cc:729] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2844b4bcc165412dbf726ec23fa1fc55, State: Initialized, Role: FOLLOWER
I20260807 06:11:56.387094 13048 tablet_bootstrap.cc:654] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:56.387208 13047 consensus_queue.cc:260] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.387493 13047 ts_tablet_manager.cc:1428] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:11:56.388165 13048 tablet_bootstrap.cc:492] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: No bootstrap required, opened a new log
I20260807 06:11:56.388250 13048 ts_tablet_manager.cc:1397] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 06:11:56.388700 13048 raft_consensus.cc:348] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.388814 13048 raft_consensus.cc:374] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:56.388845 13048 raft_consensus.cc:729] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Initialized, Role: FOLLOWER
I20260807 06:11:56.388963 13048 consensus_queue.cc:260] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.389231 13048 ts_tablet_manager.cc:1428] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:11:56.390410 12156 catalog_manager.cc:4841] TS 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130:43583): tablet 807048d6ca6e4e5fb89d2a45704e557c (table test-table [id=9eabfccd33cb4c31ab6d5064379faab0]) successfully deleted
I20260807 06:11:56.390810 13044 ts_tablet_manager.cc:1905] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:56.390971 13053 tablet_bootstrap.cc:492] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Bootstrap starting.
I20260807 06:11:56.392861 13053 tablet_bootstrap.cc:654] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Neither blocks nor log segments found. Creating new log.
I20260807 06:11:56.393292 13044 ts_tablet_manager.cc:1918] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:11:56.393438 13044 log.cc:1196] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/807048d6ca6e4e5fb89d2a45704e557c
I20260807 06:11:56.393805 13044 ts_tablet_manager.cc:1939] T 807048d6ca6e4e5fb89d2a45704e557c P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting consensus metadata
I20260807 06:11:56.394632 12156 catalog_manager.cc:4841] TS 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413): tablet 807048d6ca6e4e5fb89d2a45704e557c (table test-table [id=9eabfccd33cb4c31ab6d5064379faab0]) successfully deleted
I20260807 06:11:56.394785 13053 tablet_bootstrap.cc:492] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: No bootstrap required, opened a new log
I20260807 06:11:56.394877 13053 ts_tablet_manager.cc:1397] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:11:56.395462 13053 raft_consensus.cc:348] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.395584 13053 raft_consensus.cc:374] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:11:56.395617 13053 raft_consensus.cc:729] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e9ba3829dd42f4bf21f71f0f6d2167, State: Initialized, Role: FOLLOWER
I20260807 06:11:56.395748 13053 consensus_queue.cc:260] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.396029 13053 ts_tablet_manager.cc:1428] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:11:56.439877 13025 raft_consensus.cc:482] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:11:56.440080 13025 raft_consensus.cc:504] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.440748 13025 leader_election.cc:288] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:56.441190 12387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1316bd74ab34ed1b1afff527d0b1ee1" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2844b4bcc165412dbf726ec23fa1fc55" is_pre_election: true
I20260807 06:11:56.441268 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1316bd74ab34ed1b1afff527d0b1ee1" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" is_pre_election: true
I20260807 06:11:56.441371 12387 raft_consensus.cc:2454] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 0.
I20260807 06:11:56.441453 12776 raft_consensus.cc:2454] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 0.
I20260807 06:11:56.441731 12489 leader_election.cc:302] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:56.442292 13025 raft_consensus.cc:2781] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:11:56.442368 13025 raft_consensus.cc:482] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:11:56.442418 13025 raft_consensus.cc:3037] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:56.443967 13025 raft_consensus.cc:504] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.444339 13025 leader_election.cc:288] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 election: Requested vote from peers 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095), 48e9ba3829dd42f4bf21f71f0f6d2167 (127.9.133.131:33413)
I20260807 06:11:56.444712 12387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1316bd74ab34ed1b1afff527d0b1ee1" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2844b4bcc165412dbf726ec23fa1fc55"
I20260807 06:11:56.444847 12387 raft_consensus.cc:3037] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:56.444815 12776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1316bd74ab34ed1b1afff527d0b1ee1" candidate_uuid: "267ff4b474c748e797e450485f1dd0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167"
I20260807 06:11:56.444954 12776 raft_consensus.cc:3037] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:11:56.446290 12387 raft_consensus.cc:2454] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 1.
I20260807 06:11:56.446310 12776 raft_consensus.cc:2454] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267ff4b474c748e797e450485f1dd0e7 in term 1.
I20260807 06:11:56.446637 12489 leader_election.cc:302] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267ff4b474c748e797e450485f1dd0e7, 2844b4bcc165412dbf726ec23fa1fc55; no voters:
I20260807 06:11:56.446805 13025 raft_consensus.cc:2781] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:11:56.447227 13025 raft_consensus.cc:686] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 1 LEADER]: Becoming Leader. State: Replica: 267ff4b474c748e797e450485f1dd0e7, State: Running, Role: LEADER
I20260807 06:11:56.447407 13025 consensus_queue.cc:237] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } }
I20260807 06:11:56.449719 12178 catalog_manager.cc:5495] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 267ff4b474c748e797e450485f1dd0e7 (127.9.133.130). New cstate: current_term: 1 leader_uuid: "267ff4b474c748e797e450485f1dd0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "267ff4b474c748e797e450485f1dd0e7" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
I20260807 06:11:56.482537 12387 raft_consensus.cc:1261] T c1316bd74ab34ed1b1afff527d0b1ee1 P 2844b4bcc165412dbf726ec23fa1fc55 [term 1 FOLLOWER]: Refusing update from remote peer 267ff4b474c748e797e450485f1dd0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:56.482892 12776 raft_consensus.cc:1261] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Refusing update from remote peer 267ff4b474c748e797e450485f1dd0e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:11:56.483096 13025 consensus_queue.cc:1035] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2844b4bcc165412dbf726ec23fa1fc55" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:11:56.483428 13025 consensus_queue.cc:1035] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48e9ba3829dd42f4bf21f71f0f6d2167" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:11:56.501487 13022 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 06:11:58.065425 12178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:58.065656 12178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:11:58.069343 12178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dc3d4ecade1433eb2646fce50ba54ec: Sending DeleteTablet for 3 replicas of tablet c1316bd74ab34ed1b1afff527d0b1ee1
I20260807 06:11:58.070233 12361 tablet_service.cc:1514] Processing DeleteTablet for tablet c1316bd74ab34ed1b1afff527d0b1ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:58 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:40006
I20260807 06:11:58.070667 12551 tablet_service.cc:1514] Processing DeleteTablet for tablet c1316bd74ab34ed1b1afff527d0b1ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:58 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:54880
I20260807 06:11:58.070834 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 12275
I20260807 06:11:58.071175 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet c1316bd74ab34ed1b1afff527d0b1ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:11:58 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44798
I20260807 06:11:58.080734 13085 tablet_replica.cc:330] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: stopping tablet replica
I20260807 06:11:58.080809 13083 tablet_replica.cc:330] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: stopping tablet replica
I20260807 06:11:58.081058 13085 raft_consensus.cc:2229] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:11:58.081118 13083 raft_consensus.cc:2229] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:11:58.081290 13085 raft_consensus.cc:2258] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:58.081555 13083 raft_consensus.cc:2258] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:11:58.082413 13085 ts_tablet_manager.cc:1905] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:11:58.082482 13083 ts_tablet_manager.cc:1905] T c1316bd74ab34ed1b1afff527d0b1ee1 P 267ff4b474c748e797e450485f1dd0e7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:11:58.091144 12156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:39095 (error 108)
I20260807 06:11:58.091477 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 12463
I20260807 06:11:58.102963 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:58.102027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39095 (local address 127.0.0.1:40024)
0807 06:11:58.102705 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:58.102819 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:39095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":419,"threads_started":1}
W20260807 06:11:58.103281 12156 catalog_manager.cc:4584] TS 2844b4bcc165412dbf726ec23fa1fc55 (127.9.133.129:39095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1316bd74ab34ed1b1afff527d0b1ee1: Network error: Client connection negotiation failed: client connection to 127.9.133.129:39095: connect: Connection refused (error 111)
I20260807 06:11:58.145359 13085 ts_tablet_manager.cc:1918] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 06:11:58.145530 13085 log.cc:1196] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/c1316bd74ab34ed1b1afff527d0b1ee1
I20260807 06:11:58.145865 13085 ts_tablet_manager.cc:1939] T c1316bd74ab34ed1b1afff527d0b1ee1 P 48e9ba3829dd42f4bf21f71f0f6d2167: Deleting consensus metadata
I20260807 06:11:58.152076 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 12661
I20260807 06:11:58.161485 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:58.159313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43583 (local address 127.0.0.1:54912)
0807 06:11:58.161231 (+ 1918us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:58.161332 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.130:43583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1764}
I20260807 06:11:58.161841 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:58.161536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39095 (local address 127.0.0.1:40028)
0807 06:11:58.161704 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:58.161761 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:39095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":157,"threads_started":1}
I20260807 06:11:58.162583 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:11:58.162352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39095 (local address 127.0.0.1:40036)
0807 06:11:58.162467 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 06:11:58.162525 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:39095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 06:11:58.205569 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 12130
2026-08-07T06:11:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7803 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:11:59 dist-test-slave-h3ft krb5kdc[13093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:11:59 dist-test-slave-h3ft krb5kdc[13093](info): set up 2 sockets
Aug 07 06:11:59 dist-test-slave-h3ft krb5kdc[13093](info): commencing operation
krb5kdc: starting...
W20260807 06:12:00.550987 9750 mini_kdc.cc:121] Time spent starting KDC: real 2.311s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:00 dist-test-slave-h3ft krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:00.704154 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:34081
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:34081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:00.841255 13122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:00.841640 13122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:00.841687 13122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:00.849303 13122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:00.849428 13122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:00.849463 13122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:00.849509 13122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:00.849545 13122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:00.856419 13122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:34081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:34081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:00.858079 13122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:00.859598 13122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:00.870810 13135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:00.870827 13132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:00.870827 13129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:00.870885 13122 server_base.cc:1034] running on GCE node
I20260807 06:12:00.871984 13122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:00.873610 13122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:00.874822 13122 hybrid_clock.cc:648] HybridClock initialized: now 1786083120874805 us; error 32 us; skew 500 ppm
Aug 07 06:12:00 dist-test-slave-h3ft krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083120, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:00.878587 13122 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:00.880004 13122 webserver.cc:460] Webserver started at http://127.9.133.190:37801/ using document root <none> and password file <none>
I20260807 06:12:00.880751 13122 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:00.880822 13122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:00.881110 13122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:00.883905 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "4365f7463dbc4b83b8c79619c4af8ae2"
format_stamp: "Formatted at 2026-08-07 06:12:00 on dist-test-slave-h3ft"
server_key: "9c60bb1ea7d5b0385b7565357622e225"
server_key_iv: "fe7d9e1579c809a5db70a0cea49cb425"
server_key_version: "encryptionkey@0"
I20260807 06:12:00.884598 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "4365f7463dbc4b83b8c79619c4af8ae2"
format_stamp: "Formatted at 2026-08-07 06:12:00 on dist-test-slave-h3ft"
server_key: "9c60bb1ea7d5b0385b7565357622e225"
server_key_iv: "fe7d9e1579c809a5db70a0cea49cb425"
server_key_version: "encryptionkey@0"
I20260807 06:12:00.889314 13122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:00.892688 13143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:00.894419 13122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:00.894575 13122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "4365f7463dbc4b83b8c79619c4af8ae2"
format_stamp: "Formatted at 2026-08-07 06:12:00 on dist-test-slave-h3ft"
server_key: "9c60bb1ea7d5b0385b7565357622e225"
server_key_iv: "fe7d9e1579c809a5db70a0cea49cb425"
server_key_version: "encryptionkey@0"
I20260807 06:12:00.894701 13122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:00.905812 13122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:00.909915 13122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:00.910181 13122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:00.919066 13122 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:34081
I20260807 06:12:00.919144 13211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:34081 every 8 connection(s)
I20260807 06:12:00.920277 13122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:00.924089 13212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:00.924566 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 13122
I20260807 06:12:00.924718 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:00.925056 9750 external_mini_cluster.cc:1428] Setting key b64a91348dff9a12715f4f1f5c08c80f
I20260807 06:12:00.933226 13212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2: Bootstrap starting.
Aug 07 06:12:00 dist-test-slave-h3ft krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:00.937016 13212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:00.938236 13212 log.cc:824] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:00.941335 13212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2: No bootstrap required, opened a new log
I20260807 06:12:00.943866 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:00.926792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.9.133.190:34081)
0807 06:12:00.927465 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:00.927486 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:00.927530 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:12:00.928339 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:00.928362 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:00.928769 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:00.929135 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:00.930320 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:00.931539 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:00.932859 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:00.933477 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:00.933638 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:00.937355 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:00.937366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:00.937392 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:00.937440 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:00.939812 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:00.940337 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:00.940345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:00.940352 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:00.940515 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:00.940805 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:00.940808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:00.940810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:00.941442 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:00.941654 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:00.941981 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 06:12:00.942360 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":274,"threads_started":1}
I20260807 06:12:00.946003 13212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34081 } }
I20260807 06:12:00.946271 13212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:00.946370 13212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4365f7463dbc4b83b8c79619c4af8ae2, State: Initialized, Role: FOLLOWER
I20260807 06:12:00.946914 13212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34081 } }
I20260807 06:12:00.947137 13212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:00.947203 13212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:00.947293 13212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:00.948498 13212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34081 } }
I20260807 06:12:00.948869 13212 leader_election.cc:302] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4365f7463dbc4b83b8c79619c4af8ae2; no voters:
I20260807 06:12:00.949199 13212 leader_election.cc:288] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:00.949419 13218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:00.949699 13218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [term 1 LEADER]: Becoming Leader. State: Replica: 4365f7463dbc4b83b8c79619c4af8ae2, State: Running, Role: LEADER
I20260807 06:12:00.950109 13218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34081 } }
I20260807 06:12:00.950824 13212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:00.952185 13221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34081 } } }
I20260807 06:12:00.952368 13221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:00.952737 13222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4365f7463dbc4b83b8c79619c4af8ae2. Latest consensus state: current_term: 1 leader_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4365f7463dbc4b83b8c79619c4af8ae2" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34081 } } }
I20260807 06:12:00.952831 13233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:00.952854 13222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:00.956465 13233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:00.964010 13233 catalog_manager.cc:1347] Generated new cluster ID: cc63e13a63944df8adfe1a1884015738
I20260807 06:12:00.964118 13233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:00.997004 13233 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:00.998692 13233 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:01.007627 13233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4365f7463dbc4b83b8c79619c4af8ae2: Generated new TSK 0
I20260807 06:12:01.008771 13233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:01.050668 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34081
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:01.191322 13250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:01.191727 13250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:01.191813 13250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:01.196125 13250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:01.196254 13250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:01.196360 13250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:01.202687 13250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:01.204237 13250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:01.205591 13250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:01.217628 13258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:01.217643 13260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:01.217643 13263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:01.218007 13250 server_base.cc:1034] running on GCE node
I20260807 06:12:01.218561 13250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:01.219516 13250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:01.220723 13250 hybrid_clock.cc:648] HybridClock initialized: now 1786083121220705 us; error 32 us; skew 500 ppm
Aug 07 06:12:01 dist-test-slave-h3ft krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:01.224454 13250 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:01.225623 13250 webserver.cc:460] Webserver started at http://127.9.133.129:39529/ using document root <none> and password file <none>
I20260807 06:12:01.226393 13250 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:01.226518 13250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:01.226818 13250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:01.229784 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "185d8f48d52d471eb7a66a3bdd6d4cd1"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "10a0a540976304d806c2af61f10d394d"
server_key_iv: "2f37ffa41afd82ecc6cd978bc630f168"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.230576 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "185d8f48d52d471eb7a66a3bdd6d4cd1"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "10a0a540976304d806c2af61f10d394d"
server_key_iv: "2f37ffa41afd82ecc6cd978bc630f168"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.235903 13250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 06:12:01.239337 13271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.241044 13250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:01.241250 13250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "185d8f48d52d471eb7a66a3bdd6d4cd1"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "10a0a540976304d806c2af61f10d394d"
server_key_iv: "2f37ffa41afd82ecc6cd978bc630f168"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.241451 13250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:01.279808 13250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:01.285033 13250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:01.285329 13250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:01.286191 13250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:01.287386 13250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:01.287472 13250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.287554 13250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:01.287611 13250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.299549 13250 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:34263
I20260807 06:12:01.299572 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:34263 every 8 connection(s)
I20260807 06:12:01.300827 13250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:01.302721 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 13250
I20260807 06:12:01.302886 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:01.303352 9750 external_mini_cluster.cc:1428] Setting key 3a8a8f6abd492ef22ce8854bdb271367
Aug 07 06:12:01 dist-test-slave-h3ft krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:01.319324 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:01.303365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:52005 (local address 127.9.133.190:34081)
0807 06:12:01.303577 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:01.303582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:01.305357 (+ 1775us) server_negotiation.cc:407] Connection header received
0807 06:12:01.306558 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:01.306561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:01.306631 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:01.306730 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:01.308346 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.309004 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.310384 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.310843 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.310991 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:01.314578 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:01.314581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:01.314584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:01.314613 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:01.316453 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:01.317223 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:01.317226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:01.317227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:01.317318 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:01.317795 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:01.317798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:01.317799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:01.318023 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:01.318106 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:01.318961 (+ 855us) server_negotiation.cc:299] Negotiation successful
0807 06:12:01.319159 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 06:12:01.320616 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:01.304133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34081 (local address 127.9.133.129:52005)
0807 06:12:01.305168 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:01.305238 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:01.306295 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:01.306976 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:01.306986 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:01.307540 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:01.308174 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:01.308191 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.309162 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.309166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:01.310239 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:01.310250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.311083 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.311087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:01.311316 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:01.312192 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:01.312210 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:01.314317 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:01.316605 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:01.316615 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:01.316637 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:01.317112 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:01.317428 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:01.317434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:01.317437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:01.317683 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:01.318144 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:01.318153 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:01.318609 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 06:12:01.318902 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 06:12:01.319348 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":222,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:01.322474 13432 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34081
I20260807 06:12:01.322825 13432 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:01.323596 13432 heartbeater.cc:511] Master 127.9.133.190:34081 requested a full tablet report, sending...
I20260807 06:12:01.325920 13163 ts_manager.cc:194] Registered new tserver with Master: 185d8f48d52d471eb7a66a3bdd6d4cd1 (127.9.133.129:34263)
I20260807 06:12:01.328001 13163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:52005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:01.343503 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34081
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:01.484839 13439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:01.485196 13439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:01.485246 13439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:01.489423 13439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:01.489500 13439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:01.489565 13439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:01.495949 13439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:01.497298 13439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:01.498634 13439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:01.510535 13445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:01.510535 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:01.510545 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:01.510682 13439 server_base.cc:1034] running on GCE node
I20260807 06:12:01.511363 13439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:01.512318 13439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:01.513516 13439 hybrid_clock.cc:648] HybridClock initialized: now 1786083121513494 us; error 36 us; skew 500 ppm
Aug 07 06:12:01 dist-test-slave-h3ft krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:01.516710 13439 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:01.517699 13439 webserver.cc:460] Webserver started at http://127.9.133.130:45577/ using document root <none> and password file <none>
I20260807 06:12:01.518348 13439 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:01.518411 13439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:01.518617 13439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:01.521158 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "08382ab309c8432cabf63ebfbce1cd82"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "13d8e577eab5a3ac312649ee4a39235e"
server_key_iv: "d5b420b4daee274f30c184d62af42ae9"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.521788 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "08382ab309c8432cabf63ebfbce1cd82"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "13d8e577eab5a3ac312649ee4a39235e"
server_key_iv: "d5b420b4daee274f30c184d62af42ae9"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.526432 13439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 06:12:01.529652 13458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.531533 13439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:01.531699 13439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "08382ab309c8432cabf63ebfbce1cd82"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "13d8e577eab5a3ac312649ee4a39235e"
server_key_iv: "d5b420b4daee274f30c184d62af42ae9"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.531849 13439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:01.553920 13439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:01.557763 13439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:01.558036 13439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:01.558825 13439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:01.559928 13439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:01.560014 13439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.560113 13439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:01.560155 13439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.571249 13439 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:35557
I20260807 06:12:01.571319 13611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:35557 every 8 connection(s)
I20260807 06:12:01.572465 13439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:01.573621 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 13439
I20260807 06:12:01.573750 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:01.574163 9750 external_mini_cluster.cc:1428] Setting key 39f2cf5dc09f89861b0c63c460130974
Aug 07 06:12:01 dist-test-slave-h3ft krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:12:01.591344 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:01.575090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:60717 (local address 127.9.133.190:34081)
0807 06:12:01.575276 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:01.575282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:01.576434 (+ 1152us) server_negotiation.cc:407] Connection header received
0807 06:12:01.578666 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:01.578669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:01.578735 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:01.578819 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:01.580494 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.581131 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.582541 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.582976 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.583163 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:01.586592 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:01.586599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:01.586602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:01.586645 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:01.588622 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:01.589257 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:01.589260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:01.589261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:01.589350 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:01.589856 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:01.589859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:01.589860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:01.590096 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:01.590184 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:01.590934 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 06:12:01.591171 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:12:01.592753 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:01.575606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34081 (local address 127.9.133.130:60717)
0807 06:12:01.576237 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:01.576303 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:01.578347 (+ 2044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:01.579166 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:01.579178 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:01.579673 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:01.580308 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:01.580325 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.581297 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.581300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:01.582388 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:01.582399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.583208 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.583212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:01.583461 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:01.584310 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:01.584330 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:01.586363 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:01.588771 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:01.588781 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:01.588808 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:01.589149 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:01.589478 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:01.589482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:01.589483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:01.589659 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:01.590214 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:01.590220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:01.590583 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 06:12:01.590796 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 06:12:01.591253 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":162,"threads_started":1}
I20260807 06:12:01.594564 13612 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34081
I20260807 06:12:01.594925 13612 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:01.595680 13612 heartbeater.cc:511] Master 127.9.133.190:34081 requested a full tablet report, sending...
I20260807 06:12:01.597239 13163 ts_manager.cc:194] Registered new tserver with Master: 08382ab309c8432cabf63ebfbce1cd82 (127.9.133.130:35557)
I20260807 06:12:01.598076 13163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:60717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:01.615669 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34081
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:01.754479 13618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:01.754904 13618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:01.754995 13618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:01.759486 13618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:01.759609 13618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:01.759716 13618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:01.765776 13618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:01.767297 13618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:01.768692 13618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:01.781180 13623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:01.781163 13626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:01.781322 13618 server_base.cc:1034] running on GCE node
W20260807 06:12:01.781173 13624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:01.782152 13618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:01.783155 13618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:01.784427 13618 hybrid_clock.cc:648] HybridClock initialized: now 1786083121784386 us; error 35 us; skew 500 ppm
Aug 07 06:12:01 dist-test-slave-h3ft krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:01.788122 13618 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:01.789345 13618 webserver.cc:460] Webserver started at http://127.9.133.131:42255/ using document root <none> and password file <none>
I20260807 06:12:01.790112 13618 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:01.790194 13618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:01.790524 13618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:01.793603 13618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "2fd1214bf7e7409da4dc563f5fb21e29"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "0a5996c7cf3c24be52738e7320b406d7"
server_key_iv: "f05ae37b858f3cf4e33596a731a4e6e7"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.794394 13618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "2fd1214bf7e7409da4dc563f5fb21e29"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "0a5996c7cf3c24be52738e7320b406d7"
server_key_iv: "f05ae37b858f3cf4e33596a731a4e6e7"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.799924 13618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 06:12:01.803300 13638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.804998 13618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:01.805260 13618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "2fd1214bf7e7409da4dc563f5fb21e29"
format_stamp: "Formatted at 2026-08-07 06:12:01 on dist-test-slave-h3ft"
server_key: "0a5996c7cf3c24be52738e7320b406d7"
server_key_iv: "f05ae37b858f3cf4e33596a731a4e6e7"
server_key_version: "encryptionkey@0"
I20260807 06:12:01.805413 13618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:01.840590 13618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:01.845551 13618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:01.845968 13618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:01.846875 13618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:01.848588 13618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:01.848685 13618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.848778 13618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:01.848851 13618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:01.862306 13618 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:42915
I20260807 06:12:01.862329 13791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:42915 every 8 connection(s)
I20260807 06:12:01.863783 13618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:01.866495 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 13618
I20260807 06:12:01.866693 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:01.867120 9750 external_mini_cluster.cc:1428] Setting key 2073bcede5160e947859a4590a9e2cfd
Aug 07 06:12:01 dist-test-slave-h3ft krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:01.881423 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:01.866579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:36701 (local address 127.9.133.190:34081)
0807 06:12:01.866779 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:01.866785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:01.868518 (+ 1733us) server_negotiation.cc:407] Connection header received
0807 06:12:01.869605 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:01.869608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:01.869674 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:01.869791 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:01.871457 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.872137 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.873417 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.873679 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.873779 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:01.876965 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:01.876969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:01.876972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:01.877005 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:01.878638 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:01.879401 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:01.879405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:01.879407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:01.879509 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:01.879859 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:01.879861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:01.879863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:01.880205 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:01.880337 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:01.881048 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 06:12:01.881222 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 06:12:01.882680 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:01.867285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34081 (local address 127.9.133.131:36701)
0807 06:12:01.868298 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:01.868382 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:01.869350 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:01.870075 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:01.870084 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:01.870541 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:01.871267 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:01.871287 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.872266 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.872268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:01.873280 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:01.873290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:01.873836 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:01.873839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:01.874008 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:01.874784 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:01.874802 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:01.876743 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:01.878820 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:01.878826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:01.878843 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:01.879281 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:01.879597 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:01.879599 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:01.879601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:01.879760 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:01.880365 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:01.880378 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:01.880751 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 06:12:01.880984 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 06:12:01.881408 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":181,"threads_started":1}
I20260807 06:12:01.884235 13792 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34081
I20260807 06:12:01.884557 13792 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:01.885234 13792 heartbeater.cc:511] Master 127.9.133.190:34081 requested a full tablet report, sending...
I20260807 06:12:01.886579 13163 ts_manager.cc:194] Registered new tserver with Master: 2fd1214bf7e7409da4dc563f5fb21e29 (127.9.133.131:42915)
I20260807 06:12:01.887424 13163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:36701
I20260807 06:12:01.894613 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:12:01.915112 13216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:01.913157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.9.133.190:34081)
0807 06:12:01.913351 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:01.913357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:01.913546 (+ 189us) server_negotiation.cc:407] Connection header received
0807 06:12:01.913716 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:01.913719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:01.913774 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:01.913857 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:01.914875 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32902: BlockingRecv error: recv got EOF from 127.0.0.1:32902 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:12:01.916465 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 13250
I20260807 06:12:01.936353 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 13439
I20260807 06:12:01.959146 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 13618
I20260807 06:12:01.985234 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 13122
2026-08-07T06:12:02Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3783 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:02 dist-test-slave-h3ft krb5kdc[13814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:02 dist-test-slave-h3ft krb5kdc[13814](info): set up 2 sockets
Aug 07 06:12:02 dist-test-slave-h3ft krb5kdc[13814](info): commencing operation
krb5kdc: starting...
W20260807 06:12:03.300069 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:03 dist-test-slave-h3ft krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:03.459980 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:38319
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:38319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:03.598731 13832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:03.599180 13832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:03.599275 13832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:03.603615 13832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:03.603745 13832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:03.603783 13832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:03.603823 13832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:03.603883 13832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:03.609921 13832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:38319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:38319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:03.611541 13832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:03.613000 13832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:03.623801 13843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:03.623831 13842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:03.623842 13845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:03.624189 13832 server_base.cc:1034] running on GCE node
I20260807 06:12:03.624907 13832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:03.626246 13832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:03.627501 13832 hybrid_clock.cc:648] HybridClock initialized: now 1786083123627478 us; error 33 us; skew 500 ppm
Aug 07 06:12:03 dist-test-slave-h3ft krb5kdc[13814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083123, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:03.630972 13832 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:03.632200 13832 webserver.cc:460] Webserver started at http://127.9.133.190:40371/ using document root <none> and password file <none>
I20260807 06:12:03.632982 13832 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:03.633096 13832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:03.633391 13832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:03.636865 13832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "9c96b9dd832d4482b737241f949bc41c"
format_stamp: "Formatted at 2026-08-07 06:12:03 on dist-test-slave-h3ft"
server_key: "43e335529917a676e343ab1c848f7ba7"
server_key_iv: "87613cd3e5f01995b42a88acd2855d1f"
server_key_version: "encryptionkey@0"
I20260807 06:12:03.637624 13832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "9c96b9dd832d4482b737241f949bc41c"
format_stamp: "Formatted at 2026-08-07 06:12:03 on dist-test-slave-h3ft"
server_key: "43e335529917a676e343ab1c848f7ba7"
server_key_iv: "87613cd3e5f01995b42a88acd2855d1f"
server_key_version: "encryptionkey@0"
I20260807 06:12:03.642892 13832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 06:12:03.646229 13858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:03.647946 13832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:03.648121 13832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "9c96b9dd832d4482b737241f949bc41c"
format_stamp: "Formatted at 2026-08-07 06:12:03 on dist-test-slave-h3ft"
server_key: "43e335529917a676e343ab1c848f7ba7"
server_key_iv: "87613cd3e5f01995b42a88acd2855d1f"
server_key_version: "encryptionkey@0"
I20260807 06:12:03.648269 13832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:03.675987 13832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:03.680369 13832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:03.680641 13832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:03.689842 13832 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:38319
I20260807 06:12:03.690287 13932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:38319 every 8 connection(s)
I20260807 06:12:03.691200 13832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:03.695083 13933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:03.700544 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 13832
I20260807 06:12:03.700691 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:03.701064 9750 external_mini_cluster.cc:1428] Setting key 69c91f78b33d8c5cc9698136aea5518d
I20260807 06:12:03.703696 13933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c: Bootstrap starting.
I20260807 06:12:03.707614 13933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:03.708829 13933 log.cc:824] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:03.712078 13933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c: No bootstrap required, opened a new log
Aug 07 06:12:03 dist-test-slave-h3ft krb5kdc[13814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:03.717522 13933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96b9dd832d4482b737241f949bc41c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38319 } }
I20260807 06:12:03.717939 13933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:03.718004 13933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c96b9dd832d4482b737241f949bc41c, State: Initialized, Role: FOLLOWER
I20260807 06:12:03.718704 13933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96b9dd832d4482b737241f949bc41c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38319 } }
I20260807 06:12:03.718874 13933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:03.718977 13933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:03.719128 13933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:03.720463 13933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96b9dd832d4482b737241f949bc41c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38319 } }
I20260807 06:12:03.720949 13933 leader_election.cc:302] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c96b9dd832d4482b737241f949bc41c; no voters:
I20260807 06:12:03.721464 13933 leader_election.cc:288] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:03.721590 13938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:03.721848 13938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [term 1 LEADER]: Becoming Leader. State: Replica: 9c96b9dd832d4482b737241f949bc41c, State: Running, Role: LEADER
I20260807 06:12:03.722252 13938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96b9dd832d4482b737241f949bc41c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38319 } }
I20260807 06:12:03.722834 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:03.703077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40458 (local address 127.9.133.190:38319)
0807 06:12:03.703697 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:03.703726 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:03.703790 (+ 64us) server_negotiation.cc:407] Connection header received
0807 06:12:03.704875 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:03.704908 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:03.705541 (+ 633us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:03.706088 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:03.707140 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:03.708892 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:03.710130 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:03.710891 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:03.711130 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:03.714681 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:03.714693 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:03.714720 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:03.714766 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:03.718334 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:03.719202 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:03.719217 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:03.719225 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:03.719447 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:03.719818 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:03.719822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:03.719824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:03.720404 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:03.720674 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:03.721043 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 06:12:03.721524 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":159,"threads_started":1}
I20260807 06:12:03.723311 13933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:03.724413 13939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c96b9dd832d4482b737241f949bc41c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96b9dd832d4482b737241f949bc41c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38319 } } }
I20260807 06:12:03.724577 13939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:03.724483 13938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c96b9dd832d4482b737241f949bc41c. Latest consensus state: current_term: 1 leader_uuid: "9c96b9dd832d4482b737241f949bc41c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96b9dd832d4482b737241f949bc41c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 38319 } } }
I20260807 06:12:03.724633 13938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:03.725070 13944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:03.727051 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:12:03.729565 13944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:03.739553 13944 catalog_manager.cc:1347] Generated new cluster ID: 5ad8f55a5d4b4244b24aca2b0d9a81d5
I20260807 06:12:03.739675 13944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:03.754051 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:03.747727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.9.133.190:38319)
0807 06:12:03.747886 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:03.747891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:03.748297 (+ 406us) server_negotiation.cc:407] Connection header received
0807 06:12:03.748505 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:03.748514 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:03.748574 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:03.748671 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:03.749335 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:03.749947 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:03.750912 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:03.751268 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:03.751390 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:03.752572 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:03.752577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:03.752592 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:03.753440 (+ 848us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:03.753483 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:03.753504 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:03.753633 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:03.753744 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 06:12:03.753836 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:12:03.763590 13944 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:03.764848 13944 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:03.772333 13944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c96b9dd832d4482b737241f949bc41c: Generated new TSK 0
I20260807 06:12:03.773528 13944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:03.779886 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 13832
2026-08-07T06:12:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1795 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:03 dist-test-slave-h3ft krb5kdc[13977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:03 dist-test-slave-h3ft krb5kdc[13977](info): set up 2 sockets
Aug 07 06:12:03 dist-test-slave-h3ft krb5kdc[13977](info): commencing operation
krb5kdc: starting...
W20260807 06:12:05.089336 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:05 dist-test-slave-h3ft krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:05.248836 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40763
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:05.390897 14003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:05.391301 14003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:05.391347 14003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:05.395650 14003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:05.395735 14003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:05.395756 14003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:05.395773 14003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:05.395789 14003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:05.401803 14003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:05.403362 14003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:05.404744 14003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:12:05.415532 14003 server_base.cc:1034] running on GCE node
W20260807 06:12:05.415409 14017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:05.415448 14010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:05.415452 14014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:05.416464 14003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:05.418001 14003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:05.419188 14003 hybrid_clock.cc:648] HybridClock initialized: now 1786083125419147 us; error 48 us; skew 500 ppm
Aug 07 06:12:05 dist-test-slave-h3ft krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:05.422816 14003 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:05.424064 14003 webserver.cc:460] Webserver started at http://127.9.133.190:46447/ using document root <none> and password file <none>
I20260807 06:12:05.424822 14003 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:05.424933 14003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:05.425228 14003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:05.427944 14003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "5b6328c95a274c33bec61526e0758279"
format_stamp: "Formatted at 2026-08-07 06:12:05 on dist-test-slave-h3ft"
server_key: "df3874151ba6da24ad3882ebab200f7f"
server_key_iv: "ee63261e86147fc3350f0c0200b19b75"
server_key_version: "encryptionkey@0"
I20260807 06:12:05.428653 14003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "5b6328c95a274c33bec61526e0758279"
format_stamp: "Formatted at 2026-08-07 06:12:05 on dist-test-slave-h3ft"
server_key: "df3874151ba6da24ad3882ebab200f7f"
server_key_iv: "ee63261e86147fc3350f0c0200b19b75"
server_key_version: "encryptionkey@0"
I20260807 06:12:05.433632 14003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:05.437034 14027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:05.438581 14003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:05.438759 14003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "5b6328c95a274c33bec61526e0758279"
format_stamp: "Formatted at 2026-08-07 06:12:05 on dist-test-slave-h3ft"
server_key: "df3874151ba6da24ad3882ebab200f7f"
server_key_iv: "ee63261e86147fc3350f0c0200b19b75"
server_key_version: "encryptionkey@0"
I20260807 06:12:05.438898 14003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:05.466481 14003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:05.471128 14003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:05.471412 14003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:05.481137 14003 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40763
I20260807 06:12:05.481509 14103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40763 every 8 connection(s)
I20260807 06:12:05.482509 14003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:05.486518 14104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:05.490413 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 14003
I20260807 06:12:05.490613 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:05.490950 9750 external_mini_cluster.cc:1428] Setting key f5125e3f318cf00e8712a8c1810a2555
I20260807 06:12:05.494516 14104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279: Bootstrap starting.
I20260807 06:12:05.498102 14104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:05.499316 14104 log.cc:824] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:05.502653 14104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279: No bootstrap required, opened a new log
Aug 07 06:12:05 dist-test-slave-h3ft krb5kdc[13977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:05.507647 14104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6328c95a274c33bec61526e0758279" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40763 } }
I20260807 06:12:05.508030 14104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:05.508117 14104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6328c95a274c33bec61526e0758279, State: Initialized, Role: FOLLOWER
I20260807 06:12:05.508795 14104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6328c95a274c33bec61526e0758279" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40763 } }
I20260807 06:12:05.508986 14104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:05.509088 14104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:05.509232 14104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:05.509724 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:05.492846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.9.133.190:40763)
0807 06:12:05.493469 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:05.493490 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:05.493541 (+ 51us) server_negotiation.cc:407] Connection header received
0807 06:12:05.494307 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:05.494329 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:05.494840 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:05.495272 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:05.496408 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:05.497596 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:05.498937 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:05.499625 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:05.499774 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:05.503671 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:05.503679 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:05.503698 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:05.503734 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:05.505722 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:05.506295 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:05.506303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:05.506311 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:05.506491 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:05.506844 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:05.506847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:05.506849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:05.507534 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:05.507797 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:05.508129 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 06:12:05.508557 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":220,"threads_started":1}
I20260807 06:12:05.510804 14104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6328c95a274c33bec61526e0758279" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40763 } }
I20260807 06:12:05.511296 14104 leader_election.cc:302] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b6328c95a274c33bec61526e0758279; no voters:
I20260807 06:12:05.511725 14104 leader_election.cc:288] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:05.511818 14115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:05.512072 14115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [term 1 LEADER]: Becoming Leader. State: Replica: 5b6328c95a274c33bec61526e0758279, State: Running, Role: LEADER
I20260807 06:12:05.512450 14115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6328c95a274c33bec61526e0758279" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40763 } }
I20260807 06:12:05.513553 14104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:05.514413 14115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b6328c95a274c33bec61526e0758279. Latest consensus state: current_term: 1 leader_uuid: "5b6328c95a274c33bec61526e0758279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6328c95a274c33bec61526e0758279" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40763 } } }
I20260807 06:12:05.514572 14115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:05.516112 14116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b6328c95a274c33bec61526e0758279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6328c95a274c33bec61526e0758279" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40763 } } }
I20260807 06:12:05.516222 14116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:05.516402 14122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:05.520552 14122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:05.528951 14122 catalog_manager.cc:1347] Generated new cluster ID: 7e9a56b9f6864ea8a883a9e50b79373e
I20260807 06:12:05.529121 14122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:05.544741 14122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:05.546324 14122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:05.554877 14122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b6328c95a274c33bec61526e0758279: Generated new TSK 0
I20260807 06:12:05.555989 14122 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:05.599174 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40763
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:05.747828 14140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:05.748274 14140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:05.748368 14140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:05.752918 14140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:05.753063 14140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:05.753173 14140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:05.759544 14140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:05.761114 14140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:05.762495 14140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:05.774953 14148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:05.774945 14146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:05.775043 14151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:05.775467 14140 server_base.cc:1034] running on GCE node
I20260807 06:12:05.775988 14140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:05.776849 14140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:05.778028 14140 hybrid_clock.cc:648] HybridClock initialized: now 1786083125777985 us; error 50 us; skew 500 ppm
Aug 07 06:12:05 dist-test-slave-h3ft krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:05.781970 14140 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:05.783331 14140 webserver.cc:460] Webserver started at http://127.9.133.129:45763/ using document root <none> and password file <none>
I20260807 06:12:05.784103 14140 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:05.784176 14140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:05.784458 14140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:05.787530 14140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "845e3eab775e46d3ba7962838f3eca22"
format_stamp: "Formatted at 2026-08-07 06:12:05 on dist-test-slave-h3ft"
server_key: "b2f8108114bb6700a257782c2d2f24de"
server_key_iv: "885a4092985327e4abbdc70f32aad7fe"
server_key_version: "encryptionkey@0"
I20260807 06:12:05.788317 14140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "845e3eab775e46d3ba7962838f3eca22"
format_stamp: "Formatted at 2026-08-07 06:12:05 on dist-test-slave-h3ft"
server_key: "b2f8108114bb6700a257782c2d2f24de"
server_key_iv: "885a4092985327e4abbdc70f32aad7fe"
server_key_version: "encryptionkey@0"
I20260807 06:12:05.793663 14140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:05.797220 14160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:05.798915 14140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:05.799269 14140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "845e3eab775e46d3ba7962838f3eca22"
format_stamp: "Formatted at 2026-08-07 06:12:05 on dist-test-slave-h3ft"
server_key: "b2f8108114bb6700a257782c2d2f24de"
server_key_iv: "885a4092985327e4abbdc70f32aad7fe"
server_key_version: "encryptionkey@0"
I20260807 06:12:05.799424 14140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:05.812467 14140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:05.817162 14140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:05.817438 14140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:05.818321 14140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:05.819671 14140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:05.819727 14140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:05.819804 14140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:05.819845 14140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:05.831964 14140 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:33121
I20260807 06:12:05.831993 14325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:33121 every 8 connection(s)
I20260807 06:12:05.833251 14140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:05.840970 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 14140
I20260807 06:12:05.841151 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:05.841553 9750 external_mini_cluster.cc:1428] Setting key 98d23aab3e914d2a887d520607050ef4
Aug 07 06:12:05 dist-test-slave-h3ft krb5kdc[13977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083125, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:05.854506 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:05.836088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:34839 (local address 127.9.133.190:40763)
0807 06:12:05.836282 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:05.836287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:05.837450 (+ 1163us) server_negotiation.cc:407] Connection header received
0807 06:12:05.839367 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:05.839371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:05.839457 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:05.839577 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:05.841730 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:05.842542 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:05.844723 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:05.845001 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:05.845108 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:05.849203 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:05.849207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:05.849210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:05.849245 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:05.851309 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:05.852206 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:05.852210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:05.852212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:05.852318 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:05.852902 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:05.852906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:05.852907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:05.853186 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:05.853287 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:05.854169 (+ 882us) server_negotiation.cc:299] Negotiation successful
0807 06:12:05.854324 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:05.856168 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:05.836529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40763 (local address 127.9.133.129:34839)
0807 06:12:05.837221 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:05.837282 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:05.839041 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:05.839850 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:05.839866 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:05.840514 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:05.841474 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:05.841503 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:05.842747 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:05.842750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:05.844564 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:05.844584 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:05.845219 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:05.845227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:05.845530 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:05.846435 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:05.846470 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:05.848926 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:05.851480 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:05.851490 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:05.851520 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:05.852042 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:05.852436 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:05.852441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:05.852446 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:05.852742 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:05.853329 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:05.853347 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:05.853814 (+ 467us) client_negotiation.cc:769] Sending connection context
0807 06:12:05.854098 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 06:12:05.854605 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":172,"threads_started":1}
I20260807 06:12:05.858017 14326 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40763
I20260807 06:12:05.858376 14326 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:05.859237 14326 heartbeater.cc:511] Master 127.9.133.190:40763 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:05.861431 14052 ts_manager.cc:194] Registered new tserver with Master: 845e3eab775e46d3ba7962838f3eca22 (127.9.133.129:33121)
I20260807 06:12:05.863312 14052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:34839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:05.883863 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40763
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:06.022354 14332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:06.022715 14332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:06.022794 14332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:06.027267 14332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:06.027360 14332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:06.027460 14332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:06.033550 14332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:06.035180 14332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:06.036553 14332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:06.048898 14339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:06.049085 14332 server_base.cc:1034] running on GCE node
W20260807 06:12:06.048905 14338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:06.049309 14346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:06.049858 14332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:06.050741 14332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:06.051970 14332 hybrid_clock.cc:648] HybridClock initialized: now 1786083126051922 us; error 66 us; skew 500 ppm
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:06.055857 14332 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:06.057163 14332 webserver.cc:460] Webserver started at http://127.9.133.130:35107/ using document root <none> and password file <none>
I20260807 06:12:06.057955 14332 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:06.058069 14332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:06.058384 14332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:06.061307 14332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "97a5a3189187440fbb36c3bafa1ae9b8"
format_stamp: "Formatted at 2026-08-07 06:12:06 on dist-test-slave-h3ft"
server_key: "ba4c53d423af9248f7e1693cf9d4f737"
server_key_iv: "e84e794434194d5e9de9a01a383c9e5b"
server_key_version: "encryptionkey@0"
I20260807 06:12:06.062055 14332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "97a5a3189187440fbb36c3bafa1ae9b8"
format_stamp: "Formatted at 2026-08-07 06:12:06 on dist-test-slave-h3ft"
server_key: "ba4c53d423af9248f7e1693cf9d4f737"
server_key_iv: "e84e794434194d5e9de9a01a383c9e5b"
server_key_version: "encryptionkey@0"
I20260807 06:12:06.067374 14332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 06:12:06.070753 14354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:06.072675 14332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:06.072885 14332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "97a5a3189187440fbb36c3bafa1ae9b8"
format_stamp: "Formatted at 2026-08-07 06:12:06 on dist-test-slave-h3ft"
server_key: "ba4c53d423af9248f7e1693cf9d4f737"
server_key_iv: "e84e794434194d5e9de9a01a383c9e5b"
server_key_version: "encryptionkey@0"
I20260807 06:12:06.073096 14332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:06.092526 14332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:06.097440 14332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:06.097765 14332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:06.098670 14332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:06.099990 14332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:06.100056 14332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:06.100139 14332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:06.100180 14332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:06.112772 14332 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:42421
I20260807 06:12:06.112797 14506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:42421 every 8 connection(s)
I20260807 06:12:06.114315 14332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:06.124650 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 14332
I20260807 06:12:06.124823 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:06.125308 9750 external_mini_cluster.cc:1428] Setting key 906679fe0985b862ddcb4316d3fedd1d
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[13977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:06.134523 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:06.116955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:50495 (local address 127.9.133.190:40763)
0807 06:12:06.117156 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:06.117162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:06.118382 (+ 1220us) server_negotiation.cc:407] Connection header received
0807 06:12:06.120306 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:06.120309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:06.120377 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:06.120468 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:06.122125 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.122746 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.124274 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.124528 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.124628 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:06.128414 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:06.128422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:06.128428 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:06.128488 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:06.130948 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:06.132065 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:06.132070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:06.132072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:06.132197 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:06.132791 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:06.132795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:06.132797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:06.133081 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:06.133194 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:06.134162 (+ 968us) server_negotiation.cc:299] Negotiation successful
0807 06:12:06.134333 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:12:06.136354 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:06.117519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40763 (local address 127.9.133.130:50495)
0807 06:12:06.118174 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:06.118253 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:06.120033 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:06.120705 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:06.120713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:06.121265 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:06.121923 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:06.121940 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.122880 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.122884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:06.124141 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:06.124153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.124750 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.124756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:06.125027 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:06.125873 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:06.125903 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:06.128164 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:06.131237 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:06.131251 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:06.131285 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:06.131874 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:06.132323 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:06.132331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:06.132335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:06.132633 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:06.133232 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:06.133245 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:06.133756 (+ 511us) client_negotiation.cc:769] Sending connection context
0807 06:12:06.134057 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 06:12:06.134597 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":146,"threads_started":1}
I20260807 06:12:06.138299 14510 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40763
I20260807 06:12:06.138681 14510 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:06.139607 14510 heartbeater.cc:511] Master 127.9.133.190:40763 requested a full tablet report, sending...
I20260807 06:12:06.141476 14052 ts_manager.cc:194] Registered new tserver with Master: 97a5a3189187440fbb36c3bafa1ae9b8 (127.9.133.130:42421)
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:12:06.142273 14052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:50495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:06.168694 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40763
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:06.309192 14517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:06.309608 14517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:06.309689 14517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:06.314348 14517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:06.314477 14517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:06.314591 14517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:06.321055 14517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:06.322602 14517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:06.324028 14517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:12:06.336654 14517 server_base.cc:1034] running on GCE node
W20260807 06:12:06.336647 14523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:06.336640 14528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:06.336998 14526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:06.337577 14517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:06.338491 14517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:06.339695 14517 hybrid_clock.cc:648] HybridClock initialized: now 1786083126339672 us; error 38 us; skew 500 ppm
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:06.343613 14517 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:06.345052 14517 webserver.cc:460] Webserver started at http://127.9.133.131:46067/ using document root <none> and password file <none>
I20260807 06:12:06.345863 14517 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:06.345980 14517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:06.346290 14517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:06.349211 14517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "06de9fe666fe47d1b4498795c7dd23ca"
format_stamp: "Formatted at 2026-08-07 06:12:06 on dist-test-slave-h3ft"
server_key: "2412d0aa9b88d0a0580f29d1ad3a9ded"
server_key_iv: "45b63a085aa164a8d01302ef64b4a16c"
server_key_version: "encryptionkey@0"
I20260807 06:12:06.349954 14517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "06de9fe666fe47d1b4498795c7dd23ca"
format_stamp: "Formatted at 2026-08-07 06:12:06 on dist-test-slave-h3ft"
server_key: "2412d0aa9b88d0a0580f29d1ad3a9ded"
server_key_iv: "45b63a085aa164a8d01302ef64b4a16c"
server_key_version: "encryptionkey@0"
I20260807 06:12:06.355121 14517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 06:12:06.358570 14537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:06.360298 14517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 06:12:06.360512 14517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "06de9fe666fe47d1b4498795c7dd23ca"
format_stamp: "Formatted at 2026-08-07 06:12:06 on dist-test-slave-h3ft"
server_key: "2412d0aa9b88d0a0580f29d1ad3a9ded"
server_key_iv: "45b63a085aa164a8d01302ef64b4a16c"
server_key_version: "encryptionkey@0"
I20260807 06:12:06.360646 14517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:06.383062 14517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:06.387377 14517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:06.387665 14517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:06.388504 14517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:06.389567 14517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:06.389628 14517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:06.389708 14517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:06.389756 14517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:06.401142 14517 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:40329
I20260807 06:12:06.401165 14698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:40329 every 8 connection(s)
I20260807 06:12:06.402343 14517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:06.409850 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 14517
I20260807 06:12:06.409997 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:06.410343 9750 external_mini_cluster.cc:1428] Setting key 0e38fa80b1a2fa8a722503fb8710b7c7
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[13977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:06.420046 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:06.404861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:46281 (local address 127.9.133.190:40763)
0807 06:12:06.405055 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:06.405059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:06.406235 (+ 1176us) server_negotiation.cc:407] Connection header received
0807 06:12:06.407923 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:06.407927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:06.408019 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:06.408135 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:06.409737 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.410566 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.412227 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.412471 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.412608 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:06.415970 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:06.415973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:06.415975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:06.416006 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:06.417514 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:06.418096 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:06.418100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:06.418101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:06.418189 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:06.418575 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:06.418579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:06.418581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:06.418802 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:06.418921 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:06.419679 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 06:12:06.419833 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:12:06.421272 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:06.405427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40763 (local address 127.9.133.131:46281)
0807 06:12:06.406038 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:06.406093 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:06.407650 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:06.408362 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:06.408371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:06.408835 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:06.409510 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:06.409535 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.410730 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.410736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:06.412090 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:06.412101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.412672 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.412676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:06.412882 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:06.413802 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:06.413821 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:06.415750 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:06.417644 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:06.417652 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:06.417672 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:06.417970 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:06.418286 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:06.418289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:06.418291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:06.418459 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:06.418917 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:06.418925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:06.419368 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 06:12:06.419625 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 06:12:06.420049 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":158,"threads_started":1}
I20260807 06:12:06.422740 14699 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40763
I20260807 06:12:06.423156 14699 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:06.423911 14699 heartbeater.cc:511] Master 127.9.133.190:40763 requested a full tablet report, sending...
I20260807 06:12:06.425556 14052 ts_manager.cc:194] Registered new tserver with Master: 06de9fe666fe47d1b4498795c7dd23ca (127.9.133.131:40329)
I20260807 06:12:06.426296 14052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:46281
I20260807 06:12:06.438925 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083126, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[13977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083126, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:06.473531 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:06.463394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.9.133.190:40763)
0807 06:12:06.463575 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:06.463579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:06.463780 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:12:06.463912 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:06.463916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:06.463984 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:06.464094 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:06.464987 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.465761 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.466709 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:06.467075 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:06.467185 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:06.470098 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:06.470102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:06.470104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:06.470142 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:06.471820 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:06.472256 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:06.472259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:06.472260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:06.472349 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:06.472629 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:06.472632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:06.472633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:06.472916 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:06.473081 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:06.473229 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 06:12:06.473355 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 06:12:06.474103 14052 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57306
I20260807 06:12:06.475337 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 14140
I20260807 06:12:06.503543 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 14332
I20260807 06:12:06.525903 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 14517
I20260807 06:12:06.551229 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 14003
2026-08-07T06:12:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2767 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[14716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[14716](info): set up 2 sockets
Aug 07 06:12:06 dist-test-slave-h3ft krb5kdc[14716](info): commencing operation
krb5kdc: starting...
W20260807 06:12:07.933276 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:08.088101 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45017
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:45017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:08.232487 14731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:08.232952 14731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:08.233047 14731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:08.237629 14731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:08.237747 14731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:08.237792 14731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:08.237833 14731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:08.237869 14731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:08.244311 14731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:45017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:08.245874 14731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:08.247354 14731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:08.258387 14739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:08.258392 14740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:08.258387 14742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:08.258898 14731 server_base.cc:1034] running on GCE node
I20260807 06:12:08.259689 14731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:08.261454 14731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:08.262691 14731 hybrid_clock.cc:648] HybridClock initialized: now 1786083128262676 us; error 40 us; skew 500 ppm
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:08.266327 14731 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:08.267704 14731 webserver.cc:460] Webserver started at http://127.9.133.190:39489/ using document root <none> and password file <none>
I20260807 06:12:08.268496 14731 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:08.268576 14731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:08.268810 14731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:08.271559 14731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "9b10cffa91bb4876a717f51776cd47e0"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "18b0ecf8435ff790fb1c67a1f042e995"
server_key_iv: "351e5f024d91cb245f792adcb444a7cb"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.272228 14731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "9b10cffa91bb4876a717f51776cd47e0"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "18b0ecf8435ff790fb1c67a1f042e995"
server_key_iv: "351e5f024d91cb245f792adcb444a7cb"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.277097 14731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:08.280195 14752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.281771 14731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:08.281896 14731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "9b10cffa91bb4876a717f51776cd47e0"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "18b0ecf8435ff790fb1c67a1f042e995"
server_key_iv: "351e5f024d91cb245f792adcb444a7cb"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.282014 14731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:08.305490 14731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:08.309706 14731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:08.309926 14731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:08.320052 14731 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:45017
I20260807 06:12:08.320407 14825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:45017 every 8 connection(s)
I20260807 06:12:08.321329 14731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:08.324995 14826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:08.329181 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 14731
I20260807 06:12:08.329339 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:08.329644 9750 external_mini_cluster.cc:1428] Setting key 329ac6d26975ddbad1364d8bda68c3bf
I20260807 06:12:08.332337 14826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0: Bootstrap starting.
I20260807 06:12:08.335510 14826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:08.336571 14826 log.cc:824] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:08.339277 14826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0: No bootstrap required, opened a new log
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:08.344089 14826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b10cffa91bb4876a717f51776cd47e0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45017 } }
I20260807 06:12:08.344455 14826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:08.344535 14826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b10cffa91bb4876a717f51776cd47e0, State: Initialized, Role: FOLLOWER
I20260807 06:12:08.345170 14826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b10cffa91bb4876a717f51776cd47e0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45017 } }
I20260807 06:12:08.345367 14826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:08.345445 14826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:08.345563 14826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:08.346958 14826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b10cffa91bb4876a717f51776cd47e0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45017 } }
I20260807 06:12:08.347414 14826 leader_election.cc:302] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b10cffa91bb4876a717f51776cd47e0; no voters:
I20260807 06:12:08.347810 14826 leader_election.cc:288] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:08.347947 14834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:08.347983 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:08.331465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.9.133.190:45017)
0807 06:12:08.331965 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:08.331985 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:08.332028 (+ 43us) server_negotiation.cc:407] Connection header received
0807 06:12:08.332806 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:08.332830 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:08.333220 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:08.333552 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:08.334606 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.335854 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.337079 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.337749 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.337878 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:08.341075 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:08.341082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:08.341102 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:08.341139 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:08.344122 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:08.344681 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:08.344689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:08.344695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:08.344852 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:08.345161 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:08.345164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:08.345165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:08.345829 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:08.346079 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:08.346296 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 06:12:08.346662 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":174,"threads_started":1}
I20260807 06:12:08.348210 14834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [term 1 LEADER]: Becoming Leader. State: Replica: 9b10cffa91bb4876a717f51776cd47e0, State: Running, Role: LEADER
I20260807 06:12:08.348585 14834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b10cffa91bb4876a717f51776cd47e0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45017 } }
I20260807 06:12:08.349361 14826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:08.349934 14834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b10cffa91bb4876a717f51776cd47e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b10cffa91bb4876a717f51776cd47e0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45017 } } }
I20260807 06:12:08.350062 14834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:08.350317 14834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b10cffa91bb4876a717f51776cd47e0. Latest consensus state: current_term: 1 leader_uuid: "9b10cffa91bb4876a717f51776cd47e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b10cffa91bb4876a717f51776cd47e0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45017 } } }
I20260807 06:12:08.350414 14834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:08.351142 14841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:08.354914 14841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:08.362311 14841 catalog_manager.cc:1347] Generated new cluster ID: dc6987e442f649679ecd342d6e054c39
I20260807 06:12:08.362423 14841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:08.370966 14841 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:08.372539 14841 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:08.377518 14841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b10cffa91bb4876a717f51776cd47e0: Generated new TSK 0
I20260807 06:12:08.378521 14841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:08.414744 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45017
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:08.554605 14865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:08.554970 14865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:08.555047 14865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:08.559402 14865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:08.559487 14865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:08.559557 14865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:08.565997 14865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:08.567503 14865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:08.568918 14865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:12:08.581332 14865 server_base.cc:1034] running on GCE node
W20260807 06:12:08.581250 14872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:08.581257 14871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:08.581250 14874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:08.582165 14865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:08.583123 14865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:08.584324 14865 hybrid_clock.cc:648] HybridClock initialized: now 1786083128584285 us; error 63 us; skew 500 ppm
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:08.587905 14865 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:08.589216 14865 webserver.cc:460] Webserver started at http://127.9.133.129:34491/ using document root <none> and password file <none>
I20260807 06:12:08.589949 14865 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:08.590061 14865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:08.590358 14865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:08.593070 14865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "e7ffd597526b4f8ea4bbcc34d1aee9b4"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "ed0319a9943ab07c2cc0db4a6a316e90"
server_key_iv: "f75cf16661f0d0941d819cabaf5df3b6"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.593828 14865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "e7ffd597526b4f8ea4bbcc34d1aee9b4"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "ed0319a9943ab07c2cc0db4a6a316e90"
server_key_iv: "f75cf16661f0d0941d819cabaf5df3b6"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.598682 14865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 06:12:08.601814 14883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.603626 14865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:12:08.603804 14865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "e7ffd597526b4f8ea4bbcc34d1aee9b4"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "ed0319a9943ab07c2cc0db4a6a316e90"
server_key_iv: "f75cf16661f0d0941d819cabaf5df3b6"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.603955 14865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:08.615475 14865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:08.619256 14865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:08.619508 14865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:08.620290 14865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:08.621367 14865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:08.621447 14865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.621528 14865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:08.621573 14865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.632603 14865 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:46847
I20260807 06:12:08.632619 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:46847 every 8 connection(s)
I20260807 06:12:08.633790 14865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:08.634723 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 14865
I20260807 06:12:08.634883 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:08.635303 9750 external_mini_cluster.cc:1428] Setting key c7293383be109a5606eaf160401b44ba
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:08.653018 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:08.636370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:33383 (local address 127.9.133.190:45017)
0807 06:12:08.636550 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:08.636554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:08.637785 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 06:12:08.640335 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:08.640347 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:08.640443 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:08.640566 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:08.642147 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.642795 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.644246 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.644518 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.644640 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:08.648308 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:08.648312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:08.648315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:08.648346 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:08.650111 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:08.650751 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:08.650755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:08.650759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:08.650851 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:08.651390 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:08.651394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:08.651395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:08.651771 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:08.651880 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:08.652702 (+ 822us) server_negotiation.cc:299] Negotiation successful
0807 06:12:08.652850 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:12:08.654285 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:08.636856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45017 (local address 127.9.133.129:33383)
0807 06:12:08.637580 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:08.637654 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:08.639958 (+ 2304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:08.640826 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:08.640840 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:08.641318 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:08.641934 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:08.641950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.643054 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.643058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:08.644092 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:08.644104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.644718 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.644722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:08.644929 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:08.645807 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:08.645828 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:08.648013 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:08.650269 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:08.650277 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:08.650305 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:08.650638 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:08.650937 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:08.650940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:08.650942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:08.651227 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:08.651894 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:08.651901 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:08.652264 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 06:12:08.652552 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 06:12:08.653049 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":215,"threads_started":1}
I20260807 06:12:08.656045 15040 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45017
I20260807 06:12:08.656355 15040 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:08.657037 15040 heartbeater.cc:511] Master 127.9.133.190:45017 requested a full tablet report, sending...
I20260807 06:12:08.659159 14769 ts_manager.cc:194] Registered new tserver with Master: e7ffd597526b4f8ea4bbcc34d1aee9b4 (127.9.133.129:46847)
I20260807 06:12:08.660595 14769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:33383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:08.672647 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45017
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:08.814109 15047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:08.814524 15047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:08.814627 15047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:08.818930 15047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:08.819063 15047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:08.819187 15047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:08.825031 15047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:08.826670 15047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:08.828125 15047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:08.840233 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:08.840233 15056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:08.840235 15062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:08.840539 15047 server_base.cc:1034] running on GCE node
I20260807 06:12:08.841143 15047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:08.842036 15047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:08.843258 15047 hybrid_clock.cc:648] HybridClock initialized: now 1786083128843234 us; error 44 us; skew 500 ppm
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:08.847962 15047 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:08.849440 15047 webserver.cc:460] Webserver started at http://127.9.133.130:42677/ using document root <none> and password file <none>
I20260807 06:12:08.850461 15047 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:08.850953 15047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:08.851302 15047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:08.854012 15047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "95a0dbd74d58425bbb9014020d00408e"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "851263ce441406a0220e18d33bcf43c4"
server_key_iv: "e18a1ccc462486c58c9ae56facda5fe7"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.854846 15047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "95a0dbd74d58425bbb9014020d00408e"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "851263ce441406a0220e18d33bcf43c4"
server_key_iv: "e18a1ccc462486c58c9ae56facda5fe7"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.859758 15047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 06:12:08.862843 15070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.864492 15047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:08.864650 15047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "95a0dbd74d58425bbb9014020d00408e"
format_stamp: "Formatted at 2026-08-07 06:12:08 on dist-test-slave-h3ft"
server_key: "851263ce441406a0220e18d33bcf43c4"
server_key_iv: "e18a1ccc462486c58c9ae56facda5fe7"
server_key_version: "encryptionkey@0"
I20260807 06:12:08.864787 15047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:08.893754 15047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:08.897727 15047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:08.897994 15047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:08.898790 15047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:08.899899 15047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:08.899988 15047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.900084 15047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:08.900128 15047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:08.911263 15047 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:34151
I20260807 06:12:08.911285 15226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:34151 every 8 connection(s)
I20260807 06:12:08.912469 15047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:08.912885 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 15047
I20260807 06:12:08.913031 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:08.913461 9750 external_mini_cluster.cc:1428] Setting key af3849e46e3e2c8a082432f911e569ee
Aug 07 06:12:08 dist-test-slave-h3ft krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083128, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:12:08.929446 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:08.914932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:39995 (local address 127.9.133.190:45017)
0807 06:12:08.915121 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:08.915126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:08.916394 (+ 1268us) server_negotiation.cc:407] Connection header received
0807 06:12:08.918004 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:08.918008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:08.918079 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:08.918192 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:08.919687 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.920283 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.921687 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.921920 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.922047 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:08.924996 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:08.925000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:08.925002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:08.925028 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:08.926732 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:08.927400 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:08.927404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:08.927408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:08.927503 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:08.927884 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:08.927888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:08.927890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:08.928181 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:08.928300 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:08.929054 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 06:12:08.929242 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 06:12:08.930775 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:08.915629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45017 (local address 127.9.133.130:39995)
0807 06:12:08.916223 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:08.916275 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:08.917761 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:08.918405 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:08.918413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:08.918845 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:08.919523 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:08.919539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.920439 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.920447 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:08.921557 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:08.921566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:08.922096 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:08.922101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:08.922296 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:08.923067 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:08.923087 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:08.924789 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:08.926874 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:08.926881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:08.926901 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:08.927284 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:08.927599 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:08.927603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:08.927605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:08.927773 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:08.928299 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:08.928308 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:08.928702 (+ 394us) client_negotiation.cc:769] Sending connection context
0807 06:12:08.928930 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 06:12:08.929284 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":166,"threads_started":1}
I20260807 06:12:08.932400 15228 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45017
I20260807 06:12:08.932778 15228 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:08.933650 15228 heartbeater.cc:511] Master 127.9.133.190:45017 requested a full tablet report, sending...
I20260807 06:12:08.935129 14769 ts_manager.cc:194] Registered new tserver with Master: 95a0dbd74d58425bbb9014020d00408e (127.9.133.130:34151)
I20260807 06:12:08.936209 14769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:39995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:08.951586 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45017
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:09.094702 15234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:09.095150 15234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:09.095203 15234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:09.099597 15234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:09.099681 15234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:09.099749 15234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:09.105849 15234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:09.107398 15234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:09.108857 15234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:09.121191 15241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:09.121176 15244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:09.121176 15242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:09.121702 15234 server_base.cc:1034] running on GCE node
I20260807 06:12:09.122228 15234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:09.123113 15234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:09.124327 15234 hybrid_clock.cc:648] HybridClock initialized: now 1786083129124309 us; error 39 us; skew 500 ppm
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:09.127961 15234 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:09.129211 15234 webserver.cc:460] Webserver started at http://127.9.133.131:38455/ using document root <none> and password file <none>
I20260807 06:12:09.129946 15234 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:09.130050 15234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:09.130342 15234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:09.133080 15234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "2ad509b9f0034285a5d9bf170b6af7ac"
format_stamp: "Formatted at 2026-08-07 06:12:09 on dist-test-slave-h3ft"
server_key: "eca9b43cad6bf9c46448f77a970e6150"
server_key_iv: "d959efa80026c991c1d8f355d6937f10"
server_key_version: "encryptionkey@0"
I20260807 06:12:09.133818 15234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "2ad509b9f0034285a5d9bf170b6af7ac"
format_stamp: "Formatted at 2026-08-07 06:12:09 on dist-test-slave-h3ft"
server_key: "eca9b43cad6bf9c46448f77a970e6150"
server_key_iv: "d959efa80026c991c1d8f355d6937f10"
server_key_version: "encryptionkey@0"
I20260807 06:12:09.138597 15234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 06:12:09.141880 15255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:09.143568 15234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:09.143729 15234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "2ad509b9f0034285a5d9bf170b6af7ac"
format_stamp: "Formatted at 2026-08-07 06:12:09 on dist-test-slave-h3ft"
server_key: "eca9b43cad6bf9c46448f77a970e6150"
server_key_iv: "d959efa80026c991c1d8f355d6937f10"
server_key_version: "encryptionkey@0"
I20260807 06:12:09.143920 15234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:09.165917 15234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:09.170339 15234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:09.170594 15234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:09.171473 15234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:09.172575 15234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:09.172669 15234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:09.172741 15234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:09.172786 15234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:09.184422 15234 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:38927
I20260807 06:12:09.184444 15407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:38927 every 8 connection(s)
I20260807 06:12:09.185680 15234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:09.192619 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 15234
I20260807 06:12:09.192821 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:09.193301 9750 external_mini_cluster.cc:1428] Setting key c6839e168741d3ee4e62dd50bd244b7a
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:09.203428 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:09.188276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:36523 (local address 127.9.133.190:45017)
0807 06:12:09.188490 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:09.188495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:09.189576 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 06:12:09.191472 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:09.191476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:09.191542 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:09.191644 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:09.193511 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.194298 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.196016 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.196341 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.196443 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:09.199595 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:09.199598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:09.199600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:09.199632 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:09.201115 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:09.201725 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:09.201728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:09.201729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:09.201817 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:09.202165 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:09.202168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:09.202169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:09.202380 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:09.202458 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:09.203140 (+ 682us) server_negotiation.cc:299] Negotiation successful
0807 06:12:09.203268 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 06:12:09.204625 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:09.188789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45017 (local address 127.9.133.131:36523)
0807 06:12:09.189377 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:09.189434 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:09.191176 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:09.191856 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:09.191864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:09.192390 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:09.193250 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:09.193269 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.194511 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.194516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:09.195855 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:09.195872 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.196517 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.196520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:09.196710 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:09.197459 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:09.197492 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:09.199398 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:09.201248 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:09.201254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:09.201271 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:09.201623 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:09.201902 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:09.201904 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:09.201906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:09.202077 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:09.202485 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:09.202492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:09.202813 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 06:12:09.203074 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 06:12:09.203446 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":150,"threads_started":1}
I20260807 06:12:09.206036 15408 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45017
I20260807 06:12:09.206360 15408 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:09.207279 15408 heartbeater.cc:511] Master 127.9.133.190:45017 requested a full tablet report, sending...
I20260807 06:12:09.208716 14769 ts_manager.cc:194] Registered new tserver with Master: 2ad509b9f0034285a5d9bf170b6af7ac (127.9.133.131:38927)
I20260807 06:12:09.209470 14769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:36523
I20260807 06:12:09.220386 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:12:09.252460 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:09.241421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.9.133.129:46847)
0807 06:12:09.241769 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:09.241779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:09.241813 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:12:09.241894 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:09.241901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:09.242168 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:09.242320 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:09.243102 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.244009 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.244874 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.245334 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.245435 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:09.248336 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:09.248348 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:09.248368 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:09.248399 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:09.250299 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:09.250841 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:09.250847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:09.250849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:09.250947 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:09.251320 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:09.251324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:09.251325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:09.251695 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:09.251927 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:09.252120 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 06:12:09.252282 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":120,"threads_started":1}
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:09.264575 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:09.256169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.9.133.190:45017)
0807 06:12:09.256313 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:09.256318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:09.256336 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:12:09.256469 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:09.256472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:09.256529 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:09.256599 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:09.257194 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.257765 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.258573 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:09.258803 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:09.258891 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:09.261405 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:09.261408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:09.261411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:09.261437 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:09.262872 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:09.263357 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:09.263359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:09.263361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:09.263442 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:09.263693 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:09.263695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:09.263696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:09.263946 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:09.264101 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:09.264316 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 06:12:09.264426 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 06:12:09.265156 14769 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026
I20260807 06:12:09.265897 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 14865
I20260807 06:12:09.280319 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 15047
I20260807 06:12:09.296439 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 15234
I20260807 06:12:09.310714 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 14731
2026-08-07T06:12:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2755 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[15425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[15425](info): set up 2 sockets
Aug 07 06:12:09 dist-test-slave-h3ft krb5kdc[15425](info): commencing operation
krb5kdc: starting...
W20260807 06:12:10.580991 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:10 dist-test-slave-h3ft krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:10.727375 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40531
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:10.872207 15442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:10.872601 15442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:10.872695 15442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:10.877059 15442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:10.877153 15442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:10.877173 15442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:10.877221 15442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:10.877256 15442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:10.883337 15442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:10.884822 15442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:10.886178 15442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:10.896275 15449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:10.896535 15442 server_base.cc:1034] running on GCE node
W20260807 06:12:10.896255 15451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:10.896260 15453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:10.897426 15442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:10.898847 15442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:10.900054 15442 hybrid_clock.cc:648] HybridClock initialized: now 1786083130900025 us; error 44 us; skew 500 ppm
Aug 07 06:12:10 dist-test-slave-h3ft krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083130, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:10.903620 15442 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:10.904497 15442 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:10.904567 15442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:10.904770 15442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:10.907419 15442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "43c471e4915a4148a1b33463ed75e80b"
format_stamp: "Formatted at 2026-08-07 06:12:10 on dist-test-slave-h3ft"
server_key: "d873f7523a89899309a318f2fba260c7"
server_key_iv: "a29c1e11c8125c61150cbab80fbbd9be"
server_key_version: "encryptionkey@0"
I20260807 06:12:10.908053 15442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "43c471e4915a4148a1b33463ed75e80b"
format_stamp: "Formatted at 2026-08-07 06:12:10 on dist-test-slave-h3ft"
server_key: "d873f7523a89899309a318f2fba260c7"
server_key_iv: "a29c1e11c8125c61150cbab80fbbd9be"
server_key_version: "encryptionkey@0"
I20260807 06:12:10.912575 15442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:10.915544 15461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:10.917129 15442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:10.917279 15442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "43c471e4915a4148a1b33463ed75e80b"
format_stamp: "Formatted at 2026-08-07 06:12:10 on dist-test-slave-h3ft"
server_key: "d873f7523a89899309a318f2fba260c7"
server_key_iv: "a29c1e11c8125c61150cbab80fbbd9be"
server_key_version: "encryptionkey@0"
I20260807 06:12:10.917402 15442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:10.929152 15442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:10.933066 15442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:10.933313 15442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:10.942298 15442 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40531
I20260807 06:12:10.942751 15533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40531 every 8 connection(s)
I20260807 06:12:10.943238 15442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:10.947176 15534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:10.947458 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 15442
I20260807 06:12:10.947625 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:10.947969 9750 external_mini_cluster.cc:1428] Setting key f259dd7810a3a3b9238932d8d1884aed
I20260807 06:12:10.955696 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b: Bootstrap starting.
I20260807 06:12:10.958519 15534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b: Neither blocks nor log segments found. Creating new log.
Aug 07 06:12:10 dist-test-slave-h3ft krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:10.959551 15534 log.cc:824] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:10.962175 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b: No bootstrap required, opened a new log
I20260807 06:12:10.965085 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:10.949786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.9.133.190:40531)
0807 06:12:10.950322 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:10.950346 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:10.950390 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:12:10.951477 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:10.951501 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:10.951902 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:10.952322 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:10.953290 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:10.954475 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:10.955598 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:10.956183 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:10.956383 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:10.959245 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:10.959251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:10.959267 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:10.959299 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:10.961305 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:10.961808 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:10.961815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:10.961825 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:10.961959 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:10.962342 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:10.962346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:10.962347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:10.962970 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:10.963225 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:10.963493 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 06:12:10.963845 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":158,"threads_started":1}
I20260807 06:12:10.967363 15534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c471e4915a4148a1b33463ed75e80b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40531 } }
I20260807 06:12:10.967669 15534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:10.967751 15534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c471e4915a4148a1b33463ed75e80b, State: Initialized, Role: FOLLOWER
I20260807 06:12:10.968282 15534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c471e4915a4148a1b33463ed75e80b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40531 } }
I20260807 06:12:10.968420 15534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:10.968472 15534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:10.968555 15534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:10.969602 15534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c471e4915a4148a1b33463ed75e80b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40531 } }
I20260807 06:12:10.969944 15534 leader_election.cc:302] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43c471e4915a4148a1b33463ed75e80b; no voters:
I20260807 06:12:10.970319 15534 leader_election.cc:288] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:10.970417 15542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:10.970649 15542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [term 1 LEADER]: Becoming Leader. State: Replica: 43c471e4915a4148a1b33463ed75e80b, State: Running, Role: LEADER
I20260807 06:12:10.970989 15542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c471e4915a4148a1b33463ed75e80b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40531 } }
I20260807 06:12:10.971825 15534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:10.973028 15545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43c471e4915a4148a1b33463ed75e80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c471e4915a4148a1b33463ed75e80b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40531 } } }
I20260807 06:12:10.973204 15545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:10.973616 15544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43c471e4915a4148a1b33463ed75e80b. Latest consensus state: current_term: 1 leader_uuid: "43c471e4915a4148a1b33463ed75e80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c471e4915a4148a1b33463ed75e80b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40531 } } }
I20260807 06:12:10.974267 15544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:10.974606 15553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:10.978149 15553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:10.985776 15553 catalog_manager.cc:1347] Generated new cluster ID: fe7b93099791482ab4aea7f9749468fa
I20260807 06:12:10.985872 15553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:10.999945 15553 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:11.001164 15553 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:11.007318 15553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b: Generated new TSK 0
I20260807 06:12:11.008179 15553 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:11.046274 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40531
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:11.184083 15567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:11.184438 15567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:11.184489 15567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:11.188656 15567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:11.188719 15567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:11.188788 15567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:11.194535 15567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:11.195901 15567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:11.197145 15567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:11.208343 15578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:11.208410 15575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:11.208392 15574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:11.208606 15567 server_base.cc:1034] running on GCE node
I20260807 06:12:11.209286 15567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:11.210114 15567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:11.211315 15567 hybrid_clock.cc:648] HybridClock initialized: now 1786083131211299 us; error 33 us; skew 500 ppm
Aug 07 06:12:11 dist-test-slave-h3ft krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083131, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:11.214792 15567 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:11.215823 15567 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:11.215932 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:11.216241 15567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:11.218814 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "f2f0136f513f40e4b9c644a020fcd321"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "af3347b71dfe841244555f0d9126a14e"
server_key_iv: "aa6c9a2b5a9e216d878ce8a33adf7138"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.219564 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "f2f0136f513f40e4b9c644a020fcd321"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "af3347b71dfe841244555f0d9126a14e"
server_key_iv: "aa6c9a2b5a9e216d878ce8a33adf7138"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.224295 15567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 06:12:11.227295 15586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.228806 15567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:11.228956 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "f2f0136f513f40e4b9c644a020fcd321"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "af3347b71dfe841244555f0d9126a14e"
server_key_iv: "aa6c9a2b5a9e216d878ce8a33adf7138"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.229095 15567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:11.246912 15567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:11.251217 15567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:11.251485 15567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:11.252295 15567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:11.253428 15567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:11.253509 15567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.253592 15567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:11.253645 15567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.264919 15567 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:33077
I20260807 06:12:11.264940 15746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:33077 every 8 connection(s)
I20260807 06:12:11.265769 15567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:11.275588 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 15567
I20260807 06:12:11.275775 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:11.276120 9750 external_mini_cluster.cc:1428] Setting key 85196d9d37d4ae386e7f7527bb0c8b64
Aug 07 06:12:11 dist-test-slave-h3ft krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083131, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:11.283334 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.268053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:56949 (local address 127.9.133.190:40531)
0807 06:12:11.268298 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.268308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.269426 (+ 1118us) server_negotiation.cc:407] Connection header received
0807 06:12:11.270934 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.270937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.271046 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.271131 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:11.273040 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.273740 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.275046 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.275325 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.275456 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.279057 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:11.279061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:11.279064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:11.279100 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:11.280877 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.281497 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.281502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.281504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.281609 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.281994 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.281998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.282000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.282221 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:11.282332 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.282940 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.283137 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:12:11.284405 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.268676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40531 (local address 127.9.133.129:56949)
0807 06:12:11.269235 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:11.269288 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:11.270695 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:11.271360 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:11.271368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:11.272028 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:11.272861 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.272891 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.273869 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.273872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.274860 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.274871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.275529 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.275534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.275752 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.276621 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:11.276646 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:11.278806 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:11.281046 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:11.281054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:11.281074 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:11.281371 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:11.281716 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:11.281720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:11.281722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:11.281887 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:11.282347 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:11.282356 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:11.282673 (+ 317us) client_negotiation.cc:769] Sending connection context
0807 06:12:11.282896 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 06:12:11.283323 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":143,"threads_started":1}
I20260807 06:12:11.285948 15747 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40531
I20260807 06:12:11.286271 15747 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:11.286849 15747 heartbeater.cc:511] Master 127.9.133.190:40531 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:11.289083 15473 ts_manager.cc:194] Registered new tserver with Master: f2f0136f513f40e4b9c644a020fcd321 (127.9.133.129:33077)
I20260807 06:12:11.290496 15473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:56949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:11.311790 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40531
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:11.446980 15752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:11.447338 15752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:11.447388 15752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:11.451596 15752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:11.451671 15752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:11.451738 15752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:11.457695 15752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:11.459157 15752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:11.460532 15752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:11.472180 15758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:11.472136 15766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:11.472182 15760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:11.472385 15752 server_base.cc:1034] running on GCE node
I20260807 06:12:11.473071 15752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:11.473979 15752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:11.475198 15752 hybrid_clock.cc:648] HybridClock initialized: now 1786083131475183 us; error 35 us; skew 500 ppm
Aug 07 06:12:11 dist-test-slave-h3ft krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083131, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:11.478595 15752 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:11.479704 15752 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:11.479780 15752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:11.479991 15752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:11.482697 15752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "972a5d56bad44b2982a78bff6e6730c6"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "d8a719e923aec635a1b7af1218d90fcc"
server_key_iv: "cd9e0192d34a3cb6328e025299e70ae2"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.483462 15752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "972a5d56bad44b2982a78bff6e6730c6"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "d8a719e923aec635a1b7af1218d90fcc"
server_key_iv: "cd9e0192d34a3cb6328e025299e70ae2"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.488215 15752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:11.491146 15774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.492707 15752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:12:11.492842 15752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "972a5d56bad44b2982a78bff6e6730c6"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "d8a719e923aec635a1b7af1218d90fcc"
server_key_iv: "cd9e0192d34a3cb6328e025299e70ae2"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.492976 15752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:11.505385 15752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:11.509109 15752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:11.509296 15752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:11.509950 15752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:11.510946 15752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:11.511099 15752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.511204 15752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:11.511245 15752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.522539 15752 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:34643
I20260807 06:12:11.522574 15934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:34643 every 8 connection(s)
I20260807 06:12:11.523406 15752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:11.531446 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 15752
I20260807 06:12:11.531611 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:11.532006 9750 external_mini_cluster.cc:1428] Setting key f28d33c30984ec1f8b9d853832f325e6
Aug 07 06:12:11 dist-test-slave-h3ft krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083131, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:11.540984 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.525603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:47855 (local address 127.9.133.190:40531)
0807 06:12:11.525768 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.525772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.526777 (+ 1005us) server_negotiation.cc:407] Connection header received
0807 06:12:11.528407 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.528412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.528493 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.528605 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:11.530092 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.530707 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.532564 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.532911 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.533092 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.536705 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:11.536710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:11.536712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:11.536742 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:11.538436 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.539088 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.539091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.539092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.539187 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.539594 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.539598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.539600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.539844 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:11.539964 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.540610 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.540779 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:12:11.542193 15937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.526025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40531 (local address 127.9.133.130:47855)
0807 06:12:11.526586 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:11.526639 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:11.528139 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:11.528820 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:11.528828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:11.529276 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:11.529910 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.529926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.530847 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.530853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.532356 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.532368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.533223 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.533231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.533413 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.534391 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:11.534416 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:11.536482 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:11.538581 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:11.538588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:11.538610 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:11.538927 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:11.539290 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:11.539295 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:11.539297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:11.539479 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:11.539962 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:11.539972 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:11.540327 (+ 355us) client_negotiation.cc:769] Sending connection context
0807 06:12:11.540561 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 06:12:11.541003 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":132,"threads_started":1}
I20260807 06:12:11.543802 15935 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40531
I20260807 06:12:11.544219 15935 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:11.544997 15935 heartbeater.cc:511] Master 127.9.133.190:40531 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:12:11.546408 15473 ts_manager.cc:194] Registered new tserver with Master: 972a5d56bad44b2982a78bff6e6730c6 (127.9.133.130:34643)
I20260807 06:12:11.547200 15473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:47855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:11.566612 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40531
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:11.704130 15941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:11.704499 15941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:11.704555 15941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:11.708863 15941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:11.708945 15941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:11.709020 15941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:11.715255 15941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:11.716676 15941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:11.717979 15941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:11.730403 15948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:11.730574 15950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:11.730403 15947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:11.730742 15941 server_base.cc:1034] running on GCE node
I20260807 06:12:11.731391 15941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:11.732254 15941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:11.733481 15941 hybrid_clock.cc:648] HybridClock initialized: now 1786083131733445 us; error 47 us; skew 500 ppm
Aug 07 06:12:11 dist-test-slave-h3ft krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083131, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:11.737223 15941 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:11.738186 15941 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:11.738296 15941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:11.738606 15941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:11.741343 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "3ba3bf2614dd4584bba97e24ce3ebee7"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "517eaa0c4d1fe2cbf010a6aefd6ca695"
server_key_iv: "590cb9177cb26d57d0ba47d4222b69ad"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.742069 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "3ba3bf2614dd4584bba97e24ce3ebee7"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "517eaa0c4d1fe2cbf010a6aefd6ca695"
server_key_iv: "590cb9177cb26d57d0ba47d4222b69ad"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.746726 15941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:12:11.749991 15959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.751816 15941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:11.752014 15941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "3ba3bf2614dd4584bba97e24ce3ebee7"
format_stamp: "Formatted at 2026-08-07 06:12:11 on dist-test-slave-h3ft"
server_key: "517eaa0c4d1fe2cbf010a6aefd6ca695"
server_key_iv: "590cb9177cb26d57d0ba47d4222b69ad"
server_key_version: "encryptionkey@0"
I20260807 06:12:11.752166 15941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:11.798259 15941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:11.802237 15941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:11.802513 15941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:11.803367 15941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:11.804471 15941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:11.804553 15941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.804642 15941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:11.804687 15941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:11.815196 15941 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:33191
I20260807 06:12:11.815239 16123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:33191 every 8 connection(s)
I20260807 06:12:11.815955 15941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:11.816599 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 15941
I20260807 06:12:11.816758 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:11.817152 9750 external_mini_cluster.cc:1428] Setting key 7b5480266735c8e1da3a8c84d7468cbf
Aug 07 06:12:11 dist-test-slave-h3ft krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083131, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:11.832717 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.818270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:53069 (local address 127.9.133.190:40531)
0807 06:12:11.818450 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.818455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.819662 (+ 1207us) server_negotiation.cc:407] Connection header received
0807 06:12:11.821233 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.821237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.821294 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.821367 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:11.822841 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.823499 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.824754 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.824981 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.825061 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.828001 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:11.828004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:11.828006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:11.828031 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:11.829416 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.830082 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.830086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.830088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.830219 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.830690 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.830696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.830699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.831364 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:11.831497 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.832323 (+ 826us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.832489 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:12:11.833889 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.818862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40531 (local address 127.9.133.131:53069)
0807 06:12:11.819488 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:11.819551 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:11.821005 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:11.821611 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:11.821618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:11.822118 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:11.822719 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.822734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.823625 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.823628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.824646 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.824655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.825135 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.825137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.825307 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.826052 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:11.826070 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:11.827820 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:11.829545 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:11.829550 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:11.829565 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:11.829929 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:11.830359 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:11.830361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:11.830363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:11.830542 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:11.831558 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:11.831567 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:11.831983 (+ 416us) client_negotiation.cc:769] Sending connection context
0807 06:12:11.832239 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 06:12:11.832694 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":222,"threads_started":1}
I20260807 06:12:11.835345 16124 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40531
I20260807 06:12:11.835680 16124 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:11.836287 16124 heartbeater.cc:511] Master 127.9.133.190:40531 requested a full tablet report, sending...
I20260807 06:12:11.837536 15473 ts_manager.cc:194] Registered new tserver with Master: 3ba3bf2614dd4584bba97e24ce3ebee7 (127.9.133.131:33191)
I20260807 06:12:11.838187 15473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:53069
I20260807 06:12:11.844594 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:12:11.855319 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.846976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.9.133.190:40531)
0807 06:12:11.847179 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.847184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.847381 (+ 197us) server_negotiation.cc:407] Connection header received
0807 06:12:11.847503 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.847505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.847592 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.847664 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:11.848499 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.849092 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.849930 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.850158 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.850240 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.851854 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:11.851858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:11.851861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:11.851893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:11.853386 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.853888 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.853891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.853892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.853988 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:11.854261 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:11.854263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:11.854264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:11.854544 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:11.854661 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.854906 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.855098 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 06:12:11.859056 15473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:11.862810 15473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:11.879660 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.874198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.9.133.131:33191)
0807 06:12:11.874813 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.874826 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.875282 (+ 456us) server_negotiation.cc:407] Connection header received
0807 06:12:11.875536 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.875541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.875701 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.875867 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:11.876087 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.877104 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.878169 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.878762 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.878893 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.879104 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.879314 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.879432 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":206,"threads_started":1}
I20260807 06:12:11.880355 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.873268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:33191 (local address 127.0.0.1:32828)
0807 06:12:11.875180 (+ 1912us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:11.875197 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:11.875360 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:11.875842 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:11.875846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:11.875865 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:11.875981 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.875987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.877241 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.877246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.878033 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.878050 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.878892 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.878896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.879072 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.879125 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 06:12:11.879265 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 06:12:11.880175 (+ 910us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1831,"thread_start_us":138,"threads_started":1}
I20260807 06:12:11.880373 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.873232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34643 (local address 127.0.0.1:47122)
0807 06:12:11.873706 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:11.873761 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:11.873906 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:11.874820 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:11.874828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:11.874858 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:11.875080 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.875088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.876868 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.876871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.878225 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.878235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.879412 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.879415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.879504 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.879529 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:12:11.879576 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 06:12:11.880197 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":519,"thread_start_us":115,"threads_started":1}
I20260807 06:12:11.880565 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.874639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33077 (local address 127.0.0.1:48048)
0807 06:12:11.875284 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:11.875298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:11.875381 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:11.876249 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:11.876252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:11.876271 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:11.876366 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.876371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.877769 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.877772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.878888 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:11.878898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.880195 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.880198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:11.880283 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.880306 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:12:11.880376 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:12:11.880448 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":103,"threads_started":1}
I20260807 06:12:11.880740 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.873495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.9.133.130:34643)
0807 06:12:11.874198 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.874212 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.874241 (+ 29us) server_negotiation.cc:407] Connection header received
0807 06:12:11.874340 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.874350 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.874610 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.875271 (+ 661us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:11.875498 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.876706 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.878383 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.879273 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.879673 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.880105 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.880441 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.880554 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":126,"threads_started":1}
I20260807 06:12:11.881369 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:11.874875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.9.133.129:33077)
0807 06:12:11.875649 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:11.875656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:11.875679 (+ 23us) server_negotiation.cc:407] Connection header received
0807 06:12:11.875767 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:11.875771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:11.875954 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:11.876101 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:11.876508 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.877630 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.879059 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:11.879900 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:11.879998 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:11.880133 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:11.880938 (+ 805us) server_negotiation.cc:299] Negotiation successful
0807 06:12:11.881050 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":149,"threads_started":1}
I20260807 06:12:11.884588 15655 tablet_service.cc:1467] Processing CreateTablet for tablet 5558e9d7f9de45818047172c69f06231 (DEFAULT_TABLE table=test-table [id=4a56252e3bf14c7b87663788788e6a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:11.884582 15842 tablet_service.cc:1467] Processing CreateTablet for tablet 5558e9d7f9de45818047172c69f06231 (DEFAULT_TABLE table=test-table [id=4a56252e3bf14c7b87663788788e6a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:11.885878 15655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5558e9d7f9de45818047172c69f06231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:11.885880 15842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5558e9d7f9de45818047172c69f06231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:11.891623 16031 tablet_service.cc:1467] Processing CreateTablet for tablet 5558e9d7f9de45818047172c69f06231 (DEFAULT_TABLE table=test-table [id=4a56252e3bf14c7b87663788788e6a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:11.893168 16031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5558e9d7f9de45818047172c69f06231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:11.893733 16150 tablet_bootstrap.cc:492] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Bootstrap starting.
I20260807 06:12:11.896936 16150 tablet_bootstrap.cc:654] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:11.898360 16150 log.cc:824] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:11.899291 16151 tablet_bootstrap.cc:492] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: Bootstrap starting.
I20260807 06:12:11.901007 16150 tablet_bootstrap.cc:492] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: No bootstrap required, opened a new log
I20260807 06:12:11.901247 16150 ts_tablet_manager.cc:1397] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 06:12:11.901230 16153 tablet_bootstrap.cc:492] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Bootstrap starting.
I20260807 06:12:11.901861 16151 tablet_bootstrap.cc:654] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:11.902911 16151 log.cc:824] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:11.903821 16153 tablet_bootstrap.cc:654] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:11.905277 16153 log.cc:824] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:11.905525 16151 tablet_bootstrap.cc:492] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: No bootstrap required, opened a new log
I20260807 06:12:11.905769 16151 ts_tablet_manager.cc:1397] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260807 06:12:11.905774 16150 raft_consensus.cc:348] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:11.906154 16150 raft_consensus.cc:374] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:11.906222 16150 raft_consensus.cc:729] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972a5d56bad44b2982a78bff6e6730c6, State: Initialized, Role: FOLLOWER
I20260807 06:12:11.906747 16150 consensus_queue.cc:260] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:11.907569 15935 heartbeater.cc:503] Master 127.9.133.190:40531 was elected leader, sending a full tablet report...
I20260807 06:12:11.907934 16150 ts_tablet_manager.cc:1428] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260807 06:12:11.908398 16153 tablet_bootstrap.cc:492] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: No bootstrap required, opened a new log
I20260807 06:12:11.908623 16153 ts_tablet_manager.cc:1397] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260807 06:12:11.910239 16151 raft_consensus.cc:348] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:11.910516 16151 raft_consensus.cc:374] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:11.910579 16151 raft_consensus.cc:729] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f0136f513f40e4b9c644a020fcd321, State: Initialized, Role: FOLLOWER
I20260807 06:12:11.911167 16151 consensus_queue.cc:260] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:11.912293 16151 ts_tablet_manager.cc:1428] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260807 06:12:11.912720 15747 heartbeater.cc:503] Master 127.9.133.190:40531 was elected leader, sending a full tablet report...
I20260807 06:12:11.912729 16153 raft_consensus.cc:348] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:11.913034 16153 raft_consensus.cc:374] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:11.913125 16153 raft_consensus.cc:729] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba3bf2614dd4584bba97e24ce3ebee7, State: Initialized, Role: FOLLOWER
I20260807 06:12:11.913635 16153 consensus_queue.cc:260] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:11.914399 16124 heartbeater.cc:503] Master 127.9.133.190:40531 was elected leader, sending a full tablet report...
I20260807 06:12:11.914814 16153 ts_tablet_manager.cc:1428] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260807 06:12:12.017553 15748 tablet.cc:2369] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:12.024744 15936 tablet.cc:2369] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:12.067322 16126 tablet.cc:2369] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:12:12.068089 16157 raft_consensus.cc:482] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:12.068415 16157 raft_consensus.cc:504] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:12.069955 16157 leader_election.cc:288] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 972a5d56bad44b2982a78bff6e6730c6 (127.9.133.130:34643), 3ba3bf2614dd4584bba97e24ce3ebee7 (127.9.133.131:33191)
I20260807 06:12:12.076478 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.070508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:51101 (local address 127.9.133.130:34643)
0807 06:12:12.070664 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:12.070670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:12.071188 (+ 518us) server_negotiation.cc:407] Connection header received
0807 06:12:12.071695 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:12.071698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:12.071749 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:12.071882 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:12.072199 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.072997 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.075108 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.075840 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.075967 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:12.076027 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:12.076221 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 06:12:12.076317 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:12:12.076511 16161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.070391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34643 (local address 127.9.133.129:51101)
0807 06:12:12.070992 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:12.071034 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:12.071220 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:12.071902 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:12.071905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:12.071924 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:12.072067 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:12.072074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.073121 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.073125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:12.074906 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:12.074921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.075972 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.075975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:12.076067 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:12.076093 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:12:12.076196 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 06:12:12.076288 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":161,"spinlock_wait_cycles":28928,"thread_start_us":126,"threads_started":1}
I20260807 06:12:12.077549 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.070391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:33191 (local address 127.9.133.129:59529)
0807 06:12:12.070992 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:12.071034 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:12.071220 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:12.071621 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:12.071628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:12.071664 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:12.071887 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:12.071900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.073121 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.073126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:12.074906 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:12.074921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.077162 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.077164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:12.077249 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:12.077270 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 06:12:12.077337 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:12:12.077403 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":157,"spinlock_wait_cycles":22272,"thread_start_us":151,"threads_started":1}
I20260807 06:12:12.077642 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.070680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:59529 (local address 127.9.133.131:33191)
0807 06:12:12.070809 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:12.070813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:12.071144 (+ 331us) server_negotiation.cc:407] Connection header received
0807 06:12:12.071386 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:12.071389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:12.071446 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:12.071534 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:12.072046 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.072981 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.076092 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.076546 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.076723 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:12.076787 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:12.077389 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 06:12:12.077493 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:12:12.077531 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5558e9d7f9de45818047172c69f06231" candidate_uuid: "f2f0136f513f40e4b9c644a020fcd321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a5d56bad44b2982a78bff6e6730c6" is_pre_election: true
I20260807 06:12:12.078030 15868 raft_consensus.cc:2454] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2f0136f513f40e4b9c644a020fcd321 in term 0.
I20260807 06:12:12.078583 16056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5558e9d7f9de45818047172c69f06231" candidate_uuid: "f2f0136f513f40e4b9c644a020fcd321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" is_pre_election: true
I20260807 06:12:12.078696 15590 leader_election.cc:302] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972a5d56bad44b2982a78bff6e6730c6, f2f0136f513f40e4b9c644a020fcd321; no voters:
I20260807 06:12:12.078994 16056 raft_consensus.cc:2454] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2f0136f513f40e4b9c644a020fcd321 in term 0.
I20260807 06:12:12.079134 16157 raft_consensus.cc:2781] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:12.079246 16157 raft_consensus.cc:482] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:12.079304 16157 raft_consensus.cc:3037] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:12.080650 16157 raft_consensus.cc:504] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:12.081073 16157 leader_election.cc:288] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [CANDIDATE]: Term 1 election: Requested vote from peers 972a5d56bad44b2982a78bff6e6730c6 (127.9.133.130:34643), 3ba3bf2614dd4584bba97e24ce3ebee7 (127.9.133.131:33191)
I20260807 06:12:12.081539 16056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5558e9d7f9de45818047172c69f06231" candidate_uuid: "f2f0136f513f40e4b9c644a020fcd321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7"
I20260807 06:12:12.081539 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5558e9d7f9de45818047172c69f06231" candidate_uuid: "f2f0136f513f40e4b9c644a020fcd321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972a5d56bad44b2982a78bff6e6730c6"
I20260807 06:12:12.081684 16056 raft_consensus.cc:3037] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:12.081703 15868 raft_consensus.cc:3037] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:12.083334 15868 raft_consensus.cc:2454] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2f0136f513f40e4b9c644a020fcd321 in term 1.
I20260807 06:12:12.083334 16056 raft_consensus.cc:2454] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2f0136f513f40e4b9c644a020fcd321 in term 1.
I20260807 06:12:12.083750 15590 leader_election.cc:302] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972a5d56bad44b2982a78bff6e6730c6, f2f0136f513f40e4b9c644a020fcd321; no voters:
I20260807 06:12:12.084072 16157 raft_consensus.cc:2781] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:12.084337 16157 raft_consensus.cc:686] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 1 LEADER]: Becoming Leader. State: Replica: f2f0136f513f40e4b9c644a020fcd321, State: Running, Role: LEADER
I20260807 06:12:12.084779 16157 consensus_queue.cc:237] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } }
I20260807 06:12:12.089188 15485 catalog_manager.cc:5495] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2f0136f513f40e4b9c644a020fcd321 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "f2f0136f513f40e4b9c644a020fcd321" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2f0136f513f40e4b9c644a020fcd321" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33077 } health_report { overall_health: HEALTHY } } }
I20260807 06:12:12.122529 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.118088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.9.133.129:33077)
0807 06:12:12.118278 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:12.118283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:12.118301 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:12:12.118406 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:12.118408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:12.118481 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:12.118590 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:12.118921 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.119532 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.120486 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:12.120711 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:12.120795 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:12.121265 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:12.121892 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:12.122142 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:12.122245 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 06:12:12.122317 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:12:12.130535 16056 raft_consensus.cc:1261] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 1 FOLLOWER]: Refusing update from remote peer f2f0136f513f40e4b9c644a020fcd321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:12.130537 15868 raft_consensus.cc:1261] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 1 FOLLOWER]: Refusing update from remote peer f2f0136f513f40e4b9c644a020fcd321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:12.131716 16157 consensus_queue.cc:1035] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972a5d56bad44b2982a78bff6e6730c6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:12.132045 16162 consensus_queue.cc:1035] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ba3bf2614dd4584bba97e24ce3ebee7" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 33191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:12.142059 16166 mvcc.cc:205] Tried to move back new op lower bound from 7315796509196099584 to 7315796509022498816. Current Snapshot: MvccSnapshot[applied={T|T < 7315796509196099584}]
I20260807 06:12:12.143399 16169 mvcc.cc:205] Tried to move back new op lower bound from 7315796509196099584 to 7315796509022498816. Current Snapshot: MvccSnapshot[applied={T|T < 7315796509196099584}]
I20260807 06:12:12.154798 15473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:12.155123 15473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:12.158879 15473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43c471e4915a4148a1b33463ed75e80b: Sending DeleteTablet for 3 replicas of tablet 5558e9d7f9de45818047172c69f06231
I20260807 06:12:12.160296 15655 tablet_service.cc:1514] Processing DeleteTablet for tablet 5558e9d7f9de45818047172c69f06231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:12 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:48048
I20260807 06:12:12.160291 15842 tablet_service.cc:1514] Processing DeleteTablet for tablet 5558e9d7f9de45818047172c69f06231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:12 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:47122
I20260807 06:12:12.160310 16031 tablet_service.cc:1514] Processing DeleteTablet for tablet 5558e9d7f9de45818047172c69f06231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:12 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:32828
I20260807 06:12:12.161047 16188 tablet_replica.cc:330] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321: stopping tablet replica
I20260807 06:12:12.161056 16189 tablet_replica.cc:330] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: stopping tablet replica
I20260807 06:12:12.161509 16189 raft_consensus.cc:2229] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:12.161509 16188 raft_consensus.cc:2229] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:12.161949 16189 raft_consensus.cc:2258] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:12.162004 16188 raft_consensus.cc:2258] T 5558e9d7f9de45818047172c69f06231 P f2f0136f513f40e4b9c644a020fcd321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:12.162473 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 15567
I20260807 06:12:12.164395 16189 ts_tablet_manager.cc:1905] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:12.167749 16189 ts_tablet_manager.cc:1918] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:12.167869 16189 log.cc:1196] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/5558e9d7f9de45818047172c69f06231
I20260807 06:12:12.168191 16189 ts_tablet_manager.cc:1939] T 5558e9d7f9de45818047172c69f06231 P 3ba3bf2614dd4584bba97e24ce3ebee7: Deleting consensus metadata
I20260807 06:12:12.170615 15462 catalog_manager.cc:4841] TS 3ba3bf2614dd4584bba97e24ce3ebee7 (127.9.133.131:33191): tablet 5558e9d7f9de45818047172c69f06231 (table test-table [id=4a56252e3bf14c7b87663788788e6a04]) successfully deleted
I20260807 06:12:12.171209 16190 tablet_replica.cc:330] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: stopping tablet replica
I20260807 06:12:12.171639 16190 raft_consensus.cc:2229] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:12.172166 16190 raft_consensus.cc:2258] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:12.174078 16190 ts_tablet_manager.cc:1905] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:12.176791 16190 ts_tablet_manager.cc:1918] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:12.176934 16190 log.cc:1196] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/5558e9d7f9de45818047172c69f06231
I20260807 06:12:12.177362 16190 ts_tablet_manager.cc:1939] T 5558e9d7f9de45818047172c69f06231 P 972a5d56bad44b2982a78bff6e6730c6: Deleting consensus metadata
W20260807 06:12:12.177385 15462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:33077 (error 108)
I20260807 06:12:12.177789 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 15752
I20260807 06:12:12.179126 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.178686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33077 (local address 127.0.0.1:48066)
0807 06:12:12.178862 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:12.178948 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 06:12:12.184625 15462 catalog_manager.cc:4584] TS f2f0136f513f40e4b9c644a020fcd321 (127.9.133.129:33077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5558e9d7f9de45818047172c69f06231: Network error: Client connection negotiation failed: client connection to 127.9.133.129:33077: connect: Connection refused (error 111)
I20260807 06:12:12.196432 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 15941
I20260807 06:12:12.197115 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.196623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34643 (local address 127.0.0.1:47136)
0807 06:12:12.196887 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:12.196970 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.130:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260807 06:12:12.212565 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 15442
2026-08-07T06:12:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2904 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T06:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:12Z Disabled control of system clock
I20260807 06:12:12.258221 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:44079
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:44079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:12:12.393103 16195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:12.393440 16195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:12.393482 16195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:12.397656 16195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:12.397737 16195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:12.397753 16195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:12.397770 16195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:12.397786 16195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:12.402802 16195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:44079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:44079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:12.404345 16195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:12.406283 16195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:12.417243 16203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:12.417209 16205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:12.417249 16207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:12.417629 16195 server_base.cc:1034] running on GCE node
I20260807 06:12:12.418449 16195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:12.420131 16195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:12.421309 16195 hybrid_clock.cc:648] HybridClock initialized: now 1786083132421277 us; error 48 us; skew 500 ppm
I20260807 06:12:12.423907 16195 webserver.cc:460] Webserver started at http://127.9.133.190:42479/ using document root <none> and password file <none>
I20260807 06:12:12.424675 16195 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:12.424746 16195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:12.425052 16195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:12.427774 16195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "cd795b4786fa4f30b28938a0d27dfb7b"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "d17fad9843fbb74cde29bb872b0c608b"
server_key_iv: "5fa0dd0188b217b31717ff1073f1d728"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.428503 16195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "cd795b4786fa4f30b28938a0d27dfb7b"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "d17fad9843fbb74cde29bb872b0c608b"
server_key_iv: "5fa0dd0188b217b31717ff1073f1d728"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.433066 16195 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:12.436179 16218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.437755 16195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:12.437958 16195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "cd795b4786fa4f30b28938a0d27dfb7b"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "d17fad9843fbb74cde29bb872b0c608b"
server_key_iv: "5fa0dd0188b217b31717ff1073f1d728"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.438117 16195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:12.443781 16195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:12.444558 16195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:12.444779 16195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:12.453591 16278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:44079 every 8 connection(s)
I20260807 06:12:12.453588 16195 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:44079
I20260807 06:12:12.454945 16195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:12.456100 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 16195
I20260807 06:12:12.456252 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:12.456624 9750 external_mini_cluster.cc:1428] Setting key fb5587b269d19d66f40391ad01264aa1
I20260807 06:12:12.459573 16279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:12.464344 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.458582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.9.133.190:44079)
0807 06:12:12.460104 (+ 1522us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:12.460124 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:12.460171 (+ 47us) server_negotiation.cc:407] Connection header received
0807 06:12:12.460996 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:12.461016 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:12.461439 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:12.461840 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:12.462838 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:12.462843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:12.462856 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:12.462867 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:12.462934 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:12.462953 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:12.463164 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:12.463240 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 06:12:12.463522 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1036,"thread_start_us":818,"threads_started":1}
I20260807 06:12:12.467325 16279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: Bootstrap starting.
I20260807 06:12:12.470268 16279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:12.471261 16279 log.cc:824] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:12.473788 16279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: No bootstrap required, opened a new log
I20260807 06:12:12.478269 16279 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44079 } }
I20260807 06:12:12.478547 16279 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:12.478612 16279 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd795b4786fa4f30b28938a0d27dfb7b, State: Initialized, Role: FOLLOWER
I20260807 06:12:12.479245 16279 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44079 } }
I20260807 06:12:12.479394 16279 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:12.479486 16279 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:12.479597 16279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:12.480720 16279 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44079 } }
I20260807 06:12:12.481144 16279 leader_election.cc:302] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd795b4786fa4f30b28938a0d27dfb7b; no voters:
I20260807 06:12:12.481528 16279 leader_election.cc:288] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:12.481729 16284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:12.482002 16284 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [term 1 LEADER]: Becoming Leader. State: Replica: cd795b4786fa4f30b28938a0d27dfb7b, State: Running, Role: LEADER
I20260807 06:12:12.482363 16284 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44079 } }
I20260807 06:12:12.483381 16279 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:12.484488 16285 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44079 } } }
I20260807 06:12:12.484558 16286 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd795b4786fa4f30b28938a0d27dfb7b. Latest consensus state: current_term: 1 leader_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd795b4786fa4f30b28938a0d27dfb7b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44079 } } }
I20260807 06:12:12.484658 16285 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:12.484663 16286 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b [sys.catalog]: This master's current role is: LEADER
W20260807 06:12:12.489590 16307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:12:12.489694 16307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:12:12.489804 16294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:12.490813 16294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:12.497078 16294 catalog_manager.cc:1347] Generated new cluster ID: a102183b18f547bb88e00733422d6519
I20260807 06:12:12.497157 16294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:12.510704 16294 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:12.511256 16294 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:12.520157 16294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: Generated new TSK 0
I20260807 06:12:12.521166 16294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:12.532205 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:44079
--builtin_ntp_servers=127.9.133.148:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:12:12.668087 16315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:12.668542 16315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:12.668632 16315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:12.673151 16315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:12.673265 16315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:12.673394 16315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:12.679328 16315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:44079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:12.681506 16315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:12.683707 16315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:12.697068 16323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:12.697137 16326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:12.697137 16324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:12.697304 16315 server_base.cc:1034] running on GCE node
I20260807 06:12:12.698558 16315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:12.699676 16315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:12.700953 16315 hybrid_clock.cc:648] HybridClock initialized: now 1786083132700923 us; error 53 us; skew 500 ppm
I20260807 06:12:12.703755 16315 webserver.cc:460] Webserver started at http://127.9.133.129:43729/ using document root <none> and password file <none>
I20260807 06:12:12.704576 16315 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:12.704689 16315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:12.704999 16315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:12.707916 16315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "9ee7216736bf418882444f2564b0194b"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "b18ed551dfbe489fee94c8c788f1bf72"
server_key_iv: "af945560c3b2395e92b06843e20903e9"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.708669 16315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "9ee7216736bf418882444f2564b0194b"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "b18ed551dfbe489fee94c8c788f1bf72"
server_key_iv: "af945560c3b2395e92b06843e20903e9"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.713506 16315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:12.716794 16334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.718361 16315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:12:12.718542 16315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "9ee7216736bf418882444f2564b0194b"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "b18ed551dfbe489fee94c8c788f1bf72"
server_key_iv: "af945560c3b2395e92b06843e20903e9"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.718701 16315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:12.724344 16315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:12.725096 16315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:12.725312 16315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:12.726004 16315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:12.727227 16315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:12.727304 16315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.727381 16315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:12.727429 16315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.750319 16315 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:43825
I20260807 06:12:12.750378 16499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:43825 every 8 connection(s)
I20260807 06:12:12.751636 16315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:12.753566 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 16315
I20260807 06:12:12.753695 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:12.754106 9750 external_mini_cluster.cc:1428] Setting key 9ba4ff7bf59462b5c4bee2eda2db9558
I20260807 06:12:12.756893 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:44079
--builtin_ntp_servers=127.9.133.148:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 06:12:12.760483 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.754332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:58625 (local address 127.9.133.190:44079)
0807 06:12:12.754546 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:12.754553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:12.756311 (+ 1758us) server_negotiation.cc:407] Connection header received
0807 06:12:12.758012 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:12.758016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:12.758070 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:12.758163 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:12.759808 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:12.759815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:12.759821 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:12.759838 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:12.759875 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:12.759889 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:12.760006 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:12.760215 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 06:12:12.760279 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 06:12:12.761608 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.755063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:44079 (local address 127.9.133.129:58625)
0807 06:12:12.756122 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:12.756199 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:12.757760 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:12.758390 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:12.758398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:12.758538 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:12.759464 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:12.759466 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:12.759524 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:12.759618 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:12.760018 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:12.760029 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:12.760041 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 06:12:12.760241 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 06:12:12.760521 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":214,"threads_started":1}
I20260807 06:12:12.763341 16501 heartbeater.cc:347] Connected to a master server at 127.9.133.190:44079
I20260807 06:12:12.763685 16501 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:12.764400 16501 heartbeater.cc:511] Master 127.9.133.190:44079 requested a full tablet report, sending...
I20260807 06:12:12.766547 16237 ts_manager.cc:194] Registered new tserver with Master: 9ee7216736bf418882444f2564b0194b (127.9.133.129:43825)
W20260807 06:12:12.903059 16507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:12.903430 16507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:12.903508 16507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:12.907876 16507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:12.907970 16507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:12.908072 16507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:12.913039 16507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:44079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:12.914484 16507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:12.916327 16507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:12.928195 16516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:12.928110 16518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:12.928161 16507 server_base.cc:1034] running on GCE node
W20260807 06:12:12.928097 16524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:12.928996 16507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:12.929718 16507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:12.930891 16507 hybrid_clock.cc:648] HybridClock initialized: now 1786083132930868 us; error 31 us; skew 500 ppm
I20260807 06:12:12.933224 16507 webserver.cc:460] Webserver started at http://127.9.133.130:37091/ using document root <none> and password file <none>
I20260807 06:12:12.933947 16507 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:12.934023 16507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:12.934307 16507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:12.936959 16507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "e654d1a23403458a9373a7dbdee88298"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "118c3a78f2c6c999792adff15f518ded"
server_key_iv: "829277e68a7d31b74cd92308c2bbfb33"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.937659 16507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "e654d1a23403458a9373a7dbdee88298"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "118c3a78f2c6c999792adff15f518ded"
server_key_iv: "829277e68a7d31b74cd92308c2bbfb33"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.942334 16507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 06:12:12.945454 16532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.947079 16507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:12:12.947223 16507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "e654d1a23403458a9373a7dbdee88298"
format_stamp: "Formatted at 2026-08-07 06:12:12 on dist-test-slave-h3ft"
server_key: "118c3a78f2c6c999792adff15f518ded"
server_key_iv: "829277e68a7d31b74cd92308c2bbfb33"
server_key_version: "encryptionkey@0"
I20260807 06:12:12.947356 16507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:12.952885 16507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:12.953629 16507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:12.953848 16507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:12.954523 16507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:12.955658 16507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:12.955737 16507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.955811 16507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:12.955853 16507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:12.966704 16507 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:34367
I20260807 06:12:12.966738 16690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:34367 every 8 connection(s)
I20260807 06:12:12.967909 16507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:12.975365 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.970241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:53679 (local address 127.9.133.190:44079)
0807 06:12:12.970410 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:12.970414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:12.971680 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 06:12:12.973207 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:12.973211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:12.973269 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:12.973388 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:12.974779 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:12.974785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:12.974792 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:12.974806 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:12.974832 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:12.974841 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:12.974956 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:12.975123 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 06:12:12.975187 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 06:12:12.976418 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:12.970654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:44079 (local address 127.9.133.130:53679)
0807 06:12:12.971491 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:12.971540 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:12.972957 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:12.973596 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:12.973604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:12.973709 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:12.974457 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:12.974459 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:12.974510 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:12.974599 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:12.974930 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:12.974940 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:12.974946 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:12:12.975180 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 06:12:12.975469 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":137,"threads_started":1}
I20260807 06:12:12.977806 16691 heartbeater.cc:347] Connected to a master server at 127.9.133.190:44079
I20260807 06:12:12.977941 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 16507
I20260807 06:12:12.978077 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:12.978118 16691 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:12.978454 9750 external_mini_cluster.cc:1428] Setting key 3ba61052d8ece3b35300f5db757ba7c7
I20260807 06:12:12.978673 16691 heartbeater.cc:511] Master 127.9.133.190:44079 requested a full tablet report, sending...
I20260807 06:12:12.980019 16236 ts_manager.cc:194] Registered new tserver with Master: e654d1a23403458a9373a7dbdee88298 (127.9.133.130:34367)
I20260807 06:12:12.980996 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:44079
--builtin_ntp_servers=127.9.133.148:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 06:12:13.115903 16695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:13.116251 16695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:13.116295 16695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:13.120703 16695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:13.120775 16695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:13.120836 16695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:13.125964 16695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:44079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:13.127476 16695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:13.129397 16695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:13.141372 16701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:13.141372 16702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:13.141372 16704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:13.141834 16695 server_base.cc:1034] running on GCE node
I20260807 06:12:13.142411 16695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:13.143359 16695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:13.144580 16695 hybrid_clock.cc:648] HybridClock initialized: now 1786083133144539 us; error 56 us; skew 500 ppm
I20260807 06:12:13.146931 16695 webserver.cc:460] Webserver started at http://127.9.133.131:41753/ using document root <none> and password file <none>
I20260807 06:12:13.147689 16695 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:13.147795 16695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:13.148082 16695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:13.150676 16695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "2333f1b93d1d44a7a361e9694f61e3d4"
format_stamp: "Formatted at 2026-08-07 06:12:13 on dist-test-slave-h3ft"
server_key: "0e116692e40030c18a1c33c934913165"
server_key_iv: "b9e3a3f66bc4e5b4493356f976481d45"
server_key_version: "encryptionkey@0"
I20260807 06:12:13.151384 16695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "2333f1b93d1d44a7a361e9694f61e3d4"
format_stamp: "Formatted at 2026-08-07 06:12:13 on dist-test-slave-h3ft"
server_key: "0e116692e40030c18a1c33c934913165"
server_key_iv: "b9e3a3f66bc4e5b4493356f976481d45"
server_key_version: "encryptionkey@0"
I20260807 06:12:13.156023 16695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:13.158857 16713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:13.160492 16695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 06:12:13.160646 16695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "2333f1b93d1d44a7a361e9694f61e3d4"
format_stamp: "Formatted at 2026-08-07 06:12:13 on dist-test-slave-h3ft"
server_key: "0e116692e40030c18a1c33c934913165"
server_key_iv: "b9e3a3f66bc4e5b4493356f976481d45"
server_key_version: "encryptionkey@0"
I20260807 06:12:13.160777 16695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:13.165946 16695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:13.166647 16695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:13.166852 16695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:13.167569 16695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:13.168644 16695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:13.168721 16695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:13.168797 16695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:13.168840 16695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:13.179352 16695 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:40847
I20260807 06:12:13.179375 16872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:40847 every 8 connection(s)
I20260807 06:12:13.180513 16695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:13.188330 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.182864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:45235 (local address 127.9.133.190:44079)
0807 06:12:13.183110 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.183122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.184653 (+ 1531us) server_negotiation.cc:407] Connection header received
0807 06:12:13.186216 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.186220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.186272 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.186370 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.187854 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.187857 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.187860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.187873 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.187898 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.187908 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.187998 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.188131 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.188179 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:12:13.189493 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.183462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:44079 (local address 127.9.133.131:45235)
0807 06:12:13.184414 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.184517 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.185958 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.186625 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.186633 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.186742 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:13.187542 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:13.187544 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:13.187596 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:13.187683 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:13.187990 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:13.187998 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:13.188003 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:12:13.188170 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 06:12:13.188474 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":182,"threads_started":1}
I20260807 06:12:13.190810 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 16695
I20260807 06:12:13.190966 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:13.190980 16873 heartbeater.cc:347] Connected to a master server at 127.9.133.190:44079
I20260807 06:12:13.191280 16873 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:13.191355 9750 external_mini_cluster.cc:1428] Setting key 243b4cb8ce2a1aeba03619e31ebb1b4f
I20260807 06:12:13.191896 16873 heartbeater.cc:511] Master 127.9.133.190:44079 requested a full tablet report, sending...
I20260807 06:12:13.193251 16236 ts_manager.cc:194] Registered new tserver with Master: 2333f1b93d1d44a7a361e9694f61e3d4 (127.9.133.131:40847)
I20260807 06:12:13.194089 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:12:13.198369 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.196441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.9.133.190:44079)
0807 06:12:13.196610 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.196615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.196836 (+ 221us) server_negotiation.cc:407] Connection header received
0807 06:12:13.197010 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.197013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.197051 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.197138 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.197930 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.197933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.197936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.197955 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.197973 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.197982 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.198077 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.198172 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.198220 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:12:13.201804 16237 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:13.205550 16237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:13.220660 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.216618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34367 (local address 127.0.0.1:51074)
0807 06:12:13.217443 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.217523 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.217657 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.218265 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.218271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.218356 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:13.219648 (+ 1292us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:13.219650 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:13.219669 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:13.219732 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:13.220230 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:13.220241 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:13.220247 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:12:13.220371 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:12:13.220470 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"spinlock_wait_cycles":9216,"thread_start_us":117,"threads_started":1}
I20260807 06:12:13.220817 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.216486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43825 (local address 127.0.0.1:50152)
0807 06:12:13.217446 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.217523 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.217657 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.219067 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.219070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.219085 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:13.219844 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:13.219849 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:13.219859 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:13.219882 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:13.220248 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:13.220250 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:13.220253 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:12:13.220370 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:12:13.220698 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":244,"spinlock_wait_cycles":4352,"thread_start_us":102,"threads_started":1}
I20260807 06:12:13.220835 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.216921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.9.133.130:34367)
0807 06:12:13.217362 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.217376 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.217797 (+ 421us) server_negotiation.cc:407] Connection header received
0807 06:12:13.217879 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.217884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.218124 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.218281 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.219909 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.219919 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.219938 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.219954 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.220010 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.220059 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.220172 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.220568 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.220642 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":203,"threads_started":1}
I20260807 06:12:13.220710 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.216685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.9.133.129:43825)
0807 06:12:13.217761 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.217771 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.217800 (+ 29us) server_negotiation.cc:407] Connection header received
0807 06:12:13.217872 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.217880 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.218121 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.218284 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.220043 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.220049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.220060 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.220073 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.220120 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.220152 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.220276 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.220401 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.220508 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":98,"threads_started":1}
I20260807 06:12:13.221298 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.218034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.9.133.131:40847)
0807 06:12:13.218575 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.218589 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.218616 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:12:13.218678 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.218686 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.218848 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.219023 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.220276 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.220282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.220295 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.220308 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.220353 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.220381 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.220486 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.221021 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.221098 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":197,"threads_started":1}
I20260807 06:12:13.220671 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.217688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:40847 (local address 127.0.0.1:49478)
0807 06:12:13.218005 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.218017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.218102 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.219295 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.219298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.219312 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:13.220012 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:13.220014 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:13.220022 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:13.220040 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:13.220478 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:13.220480 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:13.220483 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:12:13.220560 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:12:13.220587 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":116,"threads_started":1}
I20260807 06:12:13.224391 16385 tablet_service.cc:1467] Processing CreateTablet for tablet 56837f7d965f43d389f4872134f8b814 (DEFAULT_TABLE table=test-table [id=86aa22f9726f47feb60c0d87b0ad23bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:13.224386 16785 tablet_service.cc:1467] Processing CreateTablet for tablet 56837f7d965f43d389f4872134f8b814 (DEFAULT_TABLE table=test-table [id=86aa22f9726f47feb60c0d87b0ad23bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:13.224392 16594 tablet_service.cc:1467] Processing CreateTablet for tablet 56837f7d965f43d389f4872134f8b814 (DEFAULT_TABLE table=test-table [id=86aa22f9726f47feb60c0d87b0ad23bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:13.225631 16785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56837f7d965f43d389f4872134f8b814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:13.225632 16594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56837f7d965f43d389f4872134f8b814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:13.225632 16385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56837f7d965f43d389f4872134f8b814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:13.233253 16893 tablet_bootstrap.cc:492] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Bootstrap starting.
I20260807 06:12:13.234433 16896 tablet_bootstrap.cc:492] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: Bootstrap starting.
I20260807 06:12:13.236152 16893 tablet_bootstrap.cc:654] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:13.236433 16896 tablet_bootstrap.cc:654] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:13.237262 16893 log.cc:824] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:13.237316 16896 log.cc:824] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:13.238145 16895 tablet_bootstrap.cc:492] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: Bootstrap starting.
I20260807 06:12:13.239671 16896 tablet_bootstrap.cc:492] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: No bootstrap required, opened a new log
I20260807 06:12:13.239704 16893 tablet_bootstrap.cc:492] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: No bootstrap required, opened a new log
I20260807 06:12:13.239909 16896 ts_tablet_manager.cc:1397] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:12:13.239908 16893 ts_tablet_manager.cc:1397] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:12:13.240832 16895 tablet_bootstrap.cc:654] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:13.241818 16895 log.cc:824] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:13.244477 16895 tablet_bootstrap.cc:492] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: No bootstrap required, opened a new log
I20260807 06:12:13.244758 16895 ts_tablet_manager.cc:1397] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260807 06:12:13.244711 16893 raft_consensus.cc:348] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.244710 16896 raft_consensus.cc:348] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.245035 16896 raft_consensus.cc:374] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:13.245038 16893 raft_consensus.cc:374] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:13.245102 16893 raft_consensus.cc:729] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2333f1b93d1d44a7a361e9694f61e3d4, State: Initialized, Role: FOLLOWER
I20260807 06:12:13.245102 16896 raft_consensus.cc:729] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e654d1a23403458a9373a7dbdee88298, State: Initialized, Role: FOLLOWER
I20260807 06:12:13.245612 16893 consensus_queue.cc:260] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.245633 16896 consensus_queue.cc:260] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.246660 16691 heartbeater.cc:503] Master 127.9.133.190:44079 was elected leader, sending a full tablet report...
I20260807 06:12:13.246798 16893 ts_tablet_manager.cc:1428] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 06:12:13.246803 16896 ts_tablet_manager.cc:1428] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 06:12:13.247314 16873 heartbeater.cc:503] Master 127.9.133.190:44079 was elected leader, sending a full tablet report...
I20260807 06:12:13.249573 16895 raft_consensus.cc:348] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.249878 16895 raft_consensus.cc:374] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:13.249948 16895 raft_consensus.cc:729] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee7216736bf418882444f2564b0194b, State: Initialized, Role: FOLLOWER
I20260807 06:12:13.250520 16895 consensus_queue.cc:260] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.251310 16501 heartbeater.cc:503] Master 127.9.133.190:44079 was elected leader, sending a full tablet report...
I20260807 06:12:13.251729 16895 ts_tablet_manager.cc:1428] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
W20260807 06:12:13.253022 16504 tablet.cc:2369] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:12:13.386262 16902 raft_consensus.cc:482] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:13.386612 16902 raft_consensus.cc:504] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.388309 16902 leader_election.cc:288] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ee7216736bf418882444f2564b0194b (127.9.133.129:43825), 2333f1b93d1d44a7a361e9694f61e3d4 (127.9.133.131:40847)
I20260807 06:12:13.390882 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.388809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:49707 (local address 127.9.133.131:40847)
0807 06:12:13.388984 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.388991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.389349 (+ 358us) server_negotiation.cc:407] Connection header received
0807 06:12:13.389452 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.389455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.389505 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.389578 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.390433 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.390437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.390440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.390455 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.390481 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.390496 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.390603 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.390673 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.390726 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:12:13.390882 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.388795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:32929 (local address 127.9.133.129:43825)
0807 06:12:13.388974 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.388980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.389006 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:12:13.389196 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.389199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.389244 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.389362 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.390385 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.390392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.390395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.390412 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.390442 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.390454 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.390566 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.390687 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.390733 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:12:13.390884 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.388645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43825 (local address 127.9.133.130:32929)
0807 06:12:13.388930 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.388948 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.389068 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.389410 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.389414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.389434 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:13.390052 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:13.390055 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:13.390071 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:13.390109 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:13.390566 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:13.390570 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:13.390574 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:12:13.390655 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:12:13.390705 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260807 06:12:13.390906 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.388685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:40847 (local address 127.9.133.130:49707)
0807 06:12:13.389227 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.389238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.389344 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.389633 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.389637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.389655 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:13.390144 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:13.390148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:13.390160 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:13.390187 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:13.390578 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:13.390580 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:13.390583 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 06:12:13.390647 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 06:12:13.390717 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":20,"thread_start_us":181,"threads_started":1}
I20260807 06:12:13.391875 16810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56837f7d965f43d389f4872134f8b814" candidate_uuid: "e654d1a23403458a9373a7dbdee88298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" is_pre_election: true
I20260807 06:12:13.391997 16430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56837f7d965f43d389f4872134f8b814" candidate_uuid: "e654d1a23403458a9373a7dbdee88298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ee7216736bf418882444f2564b0194b" is_pre_election: true
I20260807 06:12:13.392307 16430 raft_consensus.cc:2454] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e654d1a23403458a9373a7dbdee88298 in term 0.
I20260807 06:12:13.392307 16810 raft_consensus.cc:2454] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e654d1a23403458a9373a7dbdee88298 in term 0.
I20260807 06:12:13.393054 16534 leader_election.cc:302] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2333f1b93d1d44a7a361e9694f61e3d4, 9ee7216736bf418882444f2564b0194b, e654d1a23403458a9373a7dbdee88298; no voters:
I20260807 06:12:13.393402 16902 raft_consensus.cc:2781] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:13.393494 16902 raft_consensus.cc:482] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:13.393545 16902 raft_consensus.cc:3037] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:13.394852 16902 raft_consensus.cc:504] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.395332 16902 leader_election.cc:288] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [CANDIDATE]: Term 1 election: Requested vote from peers 9ee7216736bf418882444f2564b0194b (127.9.133.129:43825), 2333f1b93d1d44a7a361e9694f61e3d4 (127.9.133.131:40847)
I20260807 06:12:13.395697 16430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56837f7d965f43d389f4872134f8b814" candidate_uuid: "e654d1a23403458a9373a7dbdee88298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ee7216736bf418882444f2564b0194b"
I20260807 06:12:13.395704 16810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56837f7d965f43d389f4872134f8b814" candidate_uuid: "e654d1a23403458a9373a7dbdee88298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2333f1b93d1d44a7a361e9694f61e3d4"
I20260807 06:12:13.395854 16430 raft_consensus.cc:3037] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:13.395864 16810 raft_consensus.cc:3037] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:13.397063 16810 raft_consensus.cc:2454] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e654d1a23403458a9373a7dbdee88298 in term 1.
I20260807 06:12:13.397150 16430 raft_consensus.cc:2454] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e654d1a23403458a9373a7dbdee88298 in term 1.
I20260807 06:12:13.397431 16534 leader_election.cc:302] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2333f1b93d1d44a7a361e9694f61e3d4, e654d1a23403458a9373a7dbdee88298; no voters:
I20260807 06:12:13.397610 16902 raft_consensus.cc:2781] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:13.397881 16902 raft_consensus.cc:686] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 1 LEADER]: Becoming Leader. State: Replica: e654d1a23403458a9373a7dbdee88298, State: Running, Role: LEADER
I20260807 06:12:13.398314 16902 consensus_queue.cc:237] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } }
I20260807 06:12:13.402161 16237 catalog_manager.cc:5495] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 reported cstate change: term changed from 0 to 1, leader changed from <none> to e654d1a23403458a9373a7dbdee88298 (127.9.133.130). New cstate: current_term: 1 leader_uuid: "e654d1a23403458a9373a7dbdee88298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e654d1a23403458a9373a7dbdee88298" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 } health_report { overall_health: UNKNOWN } } }
I20260807 06:12:13.412434 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.410658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51080 (local address 127.9.133.130:34367)
0807 06:12:13.410815 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.410820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.410834 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:13.411053 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.411057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.411142 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.411241 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.411949 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.411953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.411955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.411967 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.411996 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.412013 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.412114 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.412221 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.412264 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:12:13.419595 16430 raft_consensus.cc:1261] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b [term 1 FOLLOWER]: Refusing update from remote peer e654d1a23403458a9373a7dbdee88298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:13.419595 16810 raft_consensus.cc:1261] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 1 FOLLOWER]: Refusing update from remote peer e654d1a23403458a9373a7dbdee88298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:13.420428 16911 consensus_queue.cc:1035] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ee7216736bf418882444f2564b0194b" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:13.420652 16902 consensus_queue.cc:1035] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2333f1b93d1d44a7a361e9694f61e3d4" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 40847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:13.431277 16921 mvcc.cc:205] Tried to move back new op lower bound from 7315796514477891584 to 7315796514402680832. Current Snapshot: MvccSnapshot[applied={T|T < 7315796514477891584}]
I20260807 06:12:13.433120 16920 mvcc.cc:205] Tried to move back new op lower bound from 7315796514477891584 to 7315796514402680832. Current Snapshot: MvccSnapshot[applied={T|T < 7315796514477891584 or (T in {7315796514477891584})}]
I20260807 06:12:13.440675 16237 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:13.440867 16237 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:13.444072 16237 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd795b4786fa4f30b28938a0d27dfb7b: Sending DeleteTablet for 3 replicas of tablet 56837f7d965f43d389f4872134f8b814
I20260807 06:12:13.445020 16785 tablet_service.cc:1514] Processing DeleteTablet for tablet 56837f7d965f43d389f4872134f8b814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:13 UTC) from {username='slave'} at 127.0.0.1:49478
I20260807 06:12:13.444993 16594 tablet_service.cc:1514] Processing DeleteTablet for tablet 56837f7d965f43d389f4872134f8b814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:13 UTC) from {username='slave'} at 127.0.0.1:51074
I20260807 06:12:13.445575 16931 tablet_replica.cc:330] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: stopping tablet replica
I20260807 06:12:13.445616 16385 tablet_service.cc:1514] Processing DeleteTablet for tablet 56837f7d965f43d389f4872134f8b814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:13 UTC) from {username='slave'} at 127.0.0.1:50152
I20260807 06:12:13.445963 16931 raft_consensus.cc:2229] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:13.446257 16933 tablet_replica.cc:330] T 56837f7d965f43d389f4872134f8b814 P 9ee7216736bf418882444f2564b0194b: stopping tablet replica
I20260807 06:12:13.446405 16931 raft_consensus.cc:2258] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:13.446727 16932 tablet_replica.cc:330] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: stopping tablet replica
I20260807 06:12:13.447111 16932 raft_consensus.cc:2229] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:13.447505 16932 raft_consensus.cc:2258] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:13.447573 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 16315
I20260807 06:12:13.453402 16932 ts_tablet_manager.cc:1905] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:13.457756 16932 ts_tablet_manager.cc:1918] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:13.457901 16932 log.cc:1196] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/56837f7d965f43d389f4872134f8b814
I20260807 06:12:13.458290 16932 ts_tablet_manager.cc:1939] T 56837f7d965f43d389f4872134f8b814 P 2333f1b93d1d44a7a361e9694f61e3d4: Deleting consensus metadata
I20260807 06:12:13.460076 16221 catalog_manager.cc:4841] TS 2333f1b93d1d44a7a361e9694f61e3d4 (127.9.133.131:40847): tablet 56837f7d965f43d389f4872134f8b814 (table test-table [id=86aa22f9726f47feb60c0d87b0ad23bb]) successfully deleted
I20260807 06:12:13.463959 16931 ts_tablet_manager.cc:1905] T 56837f7d965f43d389f4872134f8b814 P e654d1a23403458a9373a7dbdee88298: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:12:13.464370 16222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:43825 (error 108)
I20260807 06:12:13.465667 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 16507
I20260807 06:12:13.468959 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.466964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43825 (local address 127.0.0.1:50162)
0807 06:12:13.468774 (+ 1810us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.468859 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1698}
W20260807 06:12:13.469148 16222 catalog_manager.cc:4584] TS 9ee7216736bf418882444f2564b0194b (127.9.133.129:43825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56837f7d965f43d389f4872134f8b814: Network error: Client connection negotiation failed: client connection to 127.9.133.129:43825: connect: Connection refused (error 111)
I20260807 06:12:13.480360 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 16695
I20260807 06:12:13.481040 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.480554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34367 (local address 127.0.0.1:51094)
0807 06:12:13.480833 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.480903 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.130:34367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 06:12:13.496104 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 16195
2026-08-07T06:12:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1281 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 06:12:13.528975 9750 mini_oidc.cc:152] Starting JWKS server
I20260807 06:12:13.531325 9750 webserver.cc:460] Webserver started at https://127.0.0.1:42415/ using document root <none> and password file <none>
I20260807 06:12:13.531539 9750 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:12:13.531896 9750 webserver.cc:460] Webserver started at http://127.0.0.1:46399/ using document root <none> and password file <none>
2026-08-07T06:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:13Z Disabled control of system clock
I20260807 06:12:13.545608 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46353
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:46353
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42415/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/testchainca.cert with env {}
W20260807 06:12:13.678097 16945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:13.678459 16945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:13.678511 16945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:13.682771 16945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:13.682844 16945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:13.682859 16945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:13.682878 16945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:13.682893 16945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:12:13.682909 16945 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:12:13.682924 16945 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42415/jwks.json
I20260807 06:12:13.688934 16945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:46353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42415/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:13.690361 16945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:13.691650 16945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:13.701980 16951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:13.701977 16952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:13.702069 16954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:13.702498 16945 server_base.cc:1034] running on GCE node
I20260807 06:12:13.703404 16945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:13.704860 16945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:13.706043 16945 hybrid_clock.cc:648] HybridClock initialized: now 1786083133706023 us; error 36 us; skew 500 ppm
I20260807 06:12:13.708506 16945 webserver.cc:460] Webserver started at http://127.9.133.190:40487/ using document root <none> and password file <none>
I20260807 06:12:13.709308 16945 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:13.709389 16945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:13.709618 16945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:13.712349 16945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "b3b7796fa19d4a98ae2c599105e601f0"
format_stamp: "Formatted at 2026-08-07 06:12:13 on dist-test-slave-h3ft"
server_key: "927e11753e1b92c89583e24846ee3018"
server_key_iv: "efb7228b7424c6fc4b23f44806b218c1"
server_key_version: "encryptionkey@0"
I20260807 06:12:13.713112 16945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "b3b7796fa19d4a98ae2c599105e601f0"
format_stamp: "Formatted at 2026-08-07 06:12:13 on dist-test-slave-h3ft"
server_key: "927e11753e1b92c89583e24846ee3018"
server_key_iv: "efb7228b7424c6fc4b23f44806b218c1"
server_key_version: "encryptionkey@0"
I20260807 06:12:13.717741 16945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:13.720711 16961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:13.722185 16945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:12:13.722304 16945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "b3b7796fa19d4a98ae2c599105e601f0"
format_stamp: "Formatted at 2026-08-07 06:12:13 on dist-test-slave-h3ft"
server_key: "927e11753e1b92c89583e24846ee3018"
server_key_iv: "efb7228b7424c6fc4b23f44806b218c1"
server_key_version: "encryptionkey@0"
I20260807 06:12:13.722411 16945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:13.743898 16945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:13.744789 16945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:13.744977 16945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:13.754060 16945 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:46353
I20260807 06:12:13.754115 17036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:46353 every 8 connection(s)
I20260807 06:12:13.755419 16945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:13.760461 17037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:13.764178 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 16945
I20260807 06:12:13.764348 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:13.764732 9750 external_mini_cluster.cc:1428] Setting key b8543b5f1431b8e2bfa9c8626cc41a32
I20260807 06:12:13.768759 17037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0: Bootstrap starting.
I20260807 06:12:13.771733 17037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:13.772902 17037 log.cc:824] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:13.776031 17037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0: No bootstrap required, opened a new log
I20260807 06:12:13.776757 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.766587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.9.133.190:46353)
0807 06:12:13.767505 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.767523 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.767567 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:12:13.768306 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.768327 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.768826 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.769335 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:13.769882 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:13.770949 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:13.772016 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:13.772846 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:13.773056 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:13.774160 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:13.774169 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:13.774190 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:13.774800 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:13.774889 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:13.774925 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:13.775071 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:13.775252 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 06:12:13.775596 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":176,"threads_started":1}
I20260807 06:12:13.780246 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:12:13.780537 17037 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7796fa19d4a98ae2c599105e601f0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46353 } }
I20260807 06:12:13.780875 17037 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:13.780951 17037 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b7796fa19d4a98ae2c599105e601f0, State: Initialized, Role: FOLLOWER
I20260807 06:12:13.781536 17037 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7796fa19d4a98ae2c599105e601f0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46353 } }
I20260807 06:12:13.781695 17037 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:13.781787 17037 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:13.781915 17037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:13.783171 17037 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7796fa19d4a98ae2c599105e601f0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46353 } }
I20260807 06:12:13.783592 17037 leader_election.cc:302] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b7796fa19d4a98ae2c599105e601f0; no voters:
I20260807 06:12:13.783973 17037 leader_election.cc:288] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:13.784132 17044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:13.784438 17044 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [term 1 LEADER]: Becoming Leader. State: Replica: b3b7796fa19d4a98ae2c599105e601f0, State: Running, Role: LEADER
I20260807 06:12:13.784835 17044 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7796fa19d4a98ae2c599105e601f0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46353 } }
I20260807 06:12:13.785596 17037 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:13.786892 17047 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b7796fa19d4a98ae2c599105e601f0. Latest consensus state: current_term: 1 leader_uuid: "b3b7796fa19d4a98ae2c599105e601f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7796fa19d4a98ae2c599105e601f0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46353 } } }
I20260807 06:12:13.787086 17047 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:13.787403 17046 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3b7796fa19d4a98ae2c599105e601f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7796fa19d4a98ae2c599105e601f0" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46353 } } }
I20260807 06:12:13.787513 17046 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:13.787657 17055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:13.791952 17055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:13.798841 17055 catalog_manager.cc:1347] Generated new cluster ID: 98b2e09209d94b66b9692448ea58b065
I20260807 06:12:13.798955 17055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:13.815034 17055 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:13.816658 17055 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:13.844551 17055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3b7796fa19d4a98ae2c599105e601f0: Generated new TSK 0
I20260807 06:12:13.845857 17055 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 06:12:13.885905 17074 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 06:12:13.886193 17074 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 06:12:13.886830 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:13.866588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.9.133.190:46353)
0807 06:12:13.866781 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:13.866786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:13.866993 (+ 207us) server_negotiation.cc:407] Connection header received
0807 06:12:13.867375 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:13.867380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:13.867458 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:13.867600 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:12:13.867849 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:13.868752 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:13.870068 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:13.870461 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:13.870582 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:13.870967 (+ 385us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:12:13.886206 (+ 15239us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 06:12:13.886640 (+ 434us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 06:12:13.886945 17041 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 06:12:13.888628 17073 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:13.866392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46353 (local address 127.0.0.1:36086)
0807 06:12:13.866889 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:13.866905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:13.867143 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:13.867608 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:13.867610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:13.867618 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:12:13.867732 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:13.867737 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:13.868967 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:13.868974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:13.869921 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:13.869931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:13.870590 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:13.870592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:13.870661 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:13.870720 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:12:13.886549 (+ 15829us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 06:12:13.886868 (+ 319us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.133.190:46353: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":154,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.133.190:46353: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 06:12:13.889552 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 16945
2026-08-07T06:12:13Z chronyd exiting
I20260807 06:12:13.936651 9750 test_util.cc:175] -----------------------------------------------
I20260807 06:12:13.936781 9750 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786083100784614-9750-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (410 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 06:12:13.938107 9750 mini_oidc.cc:152] Starting JWKS server
I20260807 06:12:13.939193 9750 webserver.cc:460] Webserver started at https://127.0.0.1:33797/ using document root <none> and password file <none>
I20260807 06:12:13.939303 9750 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:12:13.939553 9750 webserver.cc:460] Webserver started at http://127.0.0.1:41985/ using document root <none> and password file <none>
2026-08-07T06:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:13Z Disabled control of system clock
I20260807 06:12:13.952766 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42167
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:40849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42167
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33797/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/testchainca.cert with env {}
W20260807 06:12:14.089469 17083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:14.089861 17083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:14.089911 17083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:14.094144 17083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:14.094255 17083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:14.094274 17083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:14.094295 17083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:14.094314 17083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:12:14.094334 17083 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:12:14.094352 17083 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33797/jwks.json
I20260807 06:12:14.100462 17083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:40849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33797/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:14.102006 17083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:14.103631 17083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:14.115494 17094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:14.115459 17091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:14.115459 17092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:14.116036 17083 server_base.cc:1034] running on GCE node
I20260807 06:12:14.116926 17083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:14.118389 17083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:14.119580 17083 hybrid_clock.cc:648] HybridClock initialized: now 1786083134119558 us; error 37 us; skew 500 ppm
I20260807 06:12:14.127761 17083 webserver.cc:460] Webserver started at http://127.9.133.190:44317/ using document root <none> and password file <none>
I20260807 06:12:14.128612 17083 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:14.128702 17083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:14.128998 17083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:14.131942 17083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "1e8ee06067e1433c94640f92bde93cfc"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "072b8d56371b3ca2472ea9a00d8ae3d0"
server_key_iv: "8fbe5ec1f39df5a8553f900e9ad20431"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.132748 17083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "1e8ee06067e1433c94640f92bde93cfc"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "072b8d56371b3ca2472ea9a00d8ae3d0"
server_key_iv: "8fbe5ec1f39df5a8553f900e9ad20431"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.137804 17083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:14.141184 17102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:14.142819 17083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:14.142954 17083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "1e8ee06067e1433c94640f92bde93cfc"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "072b8d56371b3ca2472ea9a00d8ae3d0"
server_key_iv: "8fbe5ec1f39df5a8553f900e9ad20431"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.143159 17083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:14.170272 17083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:14.171288 17083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:14.171489 17083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:14.180868 17083 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:42167
I20260807 06:12:14.180876 17172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:42167 every 8 connection(s)
I20260807 06:12:14.182155 17083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:14.186182 17173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:14.191498 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 17083
I20260807 06:12:14.191623 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:14.191946 9750 external_mini_cluster.cc:1428] Setting key 2d01a77c1d3116886d04838a27a0c9fa
I20260807 06:12:14.193601 17173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc: Bootstrap starting.
I20260807 06:12:14.196527 17173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:14.197438 17173 log.cc:824] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:14.199738 17173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc: No bootstrap required, opened a new log
I20260807 06:12:14.202759 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:14.193668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.9.133.190:42167)
0807 06:12:14.194464 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:14.194483 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:14.194527 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:12:14.195332 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:14.195355 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:14.195833 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:14.196248 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:14.196773 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.197912 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.198876 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.199475 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.199656 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.200731 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:14.200737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:14.200752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:14.201216 (+ 464us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:14.201281 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:14.201303 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:14.201420 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:14.201600 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 06:12:14.201903 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":174,"threads_started":1}
I20260807 06:12:14.204370 17173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e8ee06067e1433c94640f92bde93cfc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42167 } }
I20260807 06:12:14.204667 17173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:14.204748 17173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e8ee06067e1433c94640f92bde93cfc, State: Initialized, Role: FOLLOWER
I20260807 06:12:14.205310 17173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e8ee06067e1433c94640f92bde93cfc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42167 } }
I20260807 06:12:14.205497 17173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:14.205571 17173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:14.205683 17173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:14.206336 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:12:14.206923 17173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e8ee06067e1433c94640f92bde93cfc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42167 } }
I20260807 06:12:14.207367 17173 leader_election.cc:302] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e8ee06067e1433c94640f92bde93cfc; no voters:
I20260807 06:12:14.207801 17173 leader_election.cc:288] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:14.207880 17180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:14.208115 17180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [term 1 LEADER]: Becoming Leader. State: Replica: 1e8ee06067e1433c94640f92bde93cfc, State: Running, Role: LEADER
I20260807 06:12:14.208459 17180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e8ee06067e1433c94640f92bde93cfc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42167 } }
I20260807 06:12:14.209340 17173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:14.210474 17183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e8ee06067e1433c94640f92bde93cfc. Latest consensus state: current_term: 1 leader_uuid: "1e8ee06067e1433c94640f92bde93cfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e8ee06067e1433c94640f92bde93cfc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42167 } } }
I20260807 06:12:14.210628 17183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:14.211122 17192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:14.211522 17181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e8ee06067e1433c94640f92bde93cfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e8ee06067e1433c94640f92bde93cfc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42167 } } }
I20260807 06:12:14.211700 17181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:14.215148 17192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:14.221711 17192 catalog_manager.cc:1347] Generated new cluster ID: e3a92cbf33884ce3a6690a17bfce6558
I20260807 06:12:14.221801 17192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:14.234889 17192 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:14.236059 17192 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:14.241654 17192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e8ee06067e1433c94640f92bde93cfc: Generated new TSK 0
I20260807 06:12:14.242501 17192 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 06:12:14.267789 17209 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 06:12:14.267928 17209 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 06:12:14.268777 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:14.251464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.9.133.190:42167)
0807 06:12:14.251632 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:14.251638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:14.251848 (+ 210us) server_negotiation.cc:407] Connection header received
0807 06:12:14.252092 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:14.252096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:14.252169 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:14.252299 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:12:14.252601 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.253251 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.254607 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.254840 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.254927 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.255327 (+ 400us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:12:14.268316 (+ 12989us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33797/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 06:12:14.268615 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33797/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 06:12:14.268882 17177 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33797/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 06:12:14.269073 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:14.251374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42167 (local address 127.0.0.1:48832)
0807 06:12:14.251740 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:14.251757 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:14.251899 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:14.252313 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:14.252317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:14.252327 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:12:14.252468 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:14.252475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.253407 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.253412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:14.254471 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:14.254483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.254968 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.254973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:14.255104 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.255137 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:12:14.268592 (+ 13455us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33797/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 06:12:14.268844 (+ 252us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.133.190:42167: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33797/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":144,"threads_started":1}
I20260807 06:12:14.270164 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 17083
2026-08-07T06:12:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (383 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 06:12:14.322160 9750 mini_oidc.cc:152] Starting JWKS server
I20260807 06:12:14.323261 9750 webserver.cc:460] Webserver started at https://127.0.0.1:44089/ using document root <none> and password file <none>
I20260807 06:12:14.323387 9750 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 06:12:14.323668 9750 webserver.cc:460] Webserver started at http://127.0.0.1:38263/ using document root <none> and password file <none>
2026-08-07T06:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:14Z Disabled control of system clock
I20260807 06:12:14.335976 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:44959
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:44959
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44089/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 06:12:14.468474 17223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:14.468860 17223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:14.468905 17223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:14.473281 17223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:14.473377 17223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:14.473393 17223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:14.473412 17223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:14.473426 17223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:12:14.473443 17223 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:12:14.473456 17223 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44089/jwks.json
I20260807 06:12:14.478693 17223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:44959
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:44959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44089/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:14.480314 17223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:14.482128 17223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:14.492471 17232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:14.492408 17230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:14.492443 17229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:14.492708 17223 server_base.cc:1034] running on GCE node
I20260807 06:12:14.493410 17223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:14.494732 17223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:14.495891 17223 hybrid_clock.cc:648] HybridClock initialized: now 1786083134495852 us; error 54 us; skew 500 ppm
I20260807 06:12:14.498096 17223 webserver.cc:460] Webserver started at http://127.9.133.190:45785/ using document root <none> and password file <none>
I20260807 06:12:14.498805 17223 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:14.498874 17223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:14.499230 17223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:14.501868 17223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "8bf921d6b5404bb3a749f529c91f93bc"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "d8c59353a329aa236d7c8cf9bee91a99"
server_key_iv: "8914ba97b5e136b4bb445bf74a3e162b"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.502549 17223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "8bf921d6b5404bb3a749f529c91f93bc"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "d8c59353a329aa236d7c8cf9bee91a99"
server_key_iv: "8914ba97b5e136b4bb445bf74a3e162b"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.507201 17223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 06:12:14.510119 17240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:14.511772 17223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:14.511938 17223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "8bf921d6b5404bb3a749f529c91f93bc"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "d8c59353a329aa236d7c8cf9bee91a99"
server_key_iv: "8914ba97b5e136b4bb445bf74a3e162b"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.512084 17223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:14.556927 17223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:14.557935 17223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:14.558187 17223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:14.567220 17223 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:44959
I20260807 06:12:14.567255 17313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:44959 every 8 connection(s)
I20260807 06:12:14.568470 17223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:14.571251 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 17223
I20260807 06:12:14.571410 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:14.571749 9750 external_mini_cluster.cc:1428] Setting key f2efb979890380094756a6d394c330b3
I20260807 06:12:14.572949 17314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:14.582052 17314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc: Bootstrap starting.
I20260807 06:12:14.584353 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:14.573412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.9.133.190:44959)
0807 06:12:14.574218 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:14.574243 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:14.574297 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:12:14.575310 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:14.575339 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:14.575934 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:14.576394 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:14.576962 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.578573 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.579674 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.580503 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.580700 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.581685 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:14.581692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:14.581714 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:14.582368 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:14.582459 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:14.582489 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:14.582600 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:14.582751 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 06:12:14.583130 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":133,"threads_started":1}
I20260807 06:12:14.585702 17314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:14.586740 17314 log.cc:824] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:14.588023 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:12:14.589496 17314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc: No bootstrap required, opened a new log
I20260807 06:12:14.594034 17314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf921d6b5404bb3a749f529c91f93bc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44959 } }
I20260807 06:12:14.594321 17314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:14.594383 17314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf921d6b5404bb3a749f529c91f93bc, State: Initialized, Role: FOLLOWER
I20260807 06:12:14.594900 17314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf921d6b5404bb3a749f529c91f93bc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44959 } }
I20260807 06:12:14.595103 17314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:14.595180 17314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:14.595314 17314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:14.596468 17314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf921d6b5404bb3a749f529c91f93bc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44959 } }
I20260807 06:12:14.596886 17314 leader_election.cc:302] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bf921d6b5404bb3a749f529c91f93bc; no voters:
I20260807 06:12:14.597270 17314 leader_election.cc:288] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:14.597473 17319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:14.597839 17319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [term 1 LEADER]: Becoming Leader. State: Replica: 8bf921d6b5404bb3a749f529c91f93bc, State: Running, Role: LEADER
I20260807 06:12:14.598243 17319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf921d6b5404bb3a749f529c91f93bc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44959 } }
I20260807 06:12:14.598753 17314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:14.600370 17321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bf921d6b5404bb3a749f529c91f93bc. Latest consensus state: current_term: 1 leader_uuid: "8bf921d6b5404bb3a749f529c91f93bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf921d6b5404bb3a749f529c91f93bc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44959 } } }
I20260807 06:12:14.600504 17321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:14.600982 17333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:14.600973 17320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bf921d6b5404bb3a749f529c91f93bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf921d6b5404bb3a749f529c91f93bc" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 44959 } } }
I20260807 06:12:14.601128 17320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:14.604784 17333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:14.611304 17333 catalog_manager.cc:1347] Generated new cluster ID: d17637f61d024ee4a7c797cbdd580a27
I20260807 06:12:14.611418 17333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:14.632367 17333 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:14.634006 17333 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:14.647338 17333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bf921d6b5404bb3a749f529c91f93bc: Generated new TSK 0
I20260807 06:12:14.648505 17333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:14.671916 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:14.665794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.9.133.190:44959)
0807 06:12:14.665977 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:14.665984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:14.666204 (+ 220us) server_negotiation.cc:407] Connection header received
0807 06:12:14.666428 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:14.666431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:14.666491 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:14.666606 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:12:14.666899 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.667609 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.668893 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.669138 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.669231 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.669609 (+ 378us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:12:14.671473 (+ 1864us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 06:12:14.671737 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47242: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 06:12:14.672015 17317 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47242: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 06:12:14.673983 17357 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:14.665706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:44959 (local address 127.0.0.1:47242)
0807 06:12:14.666092 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:14.666109 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:14.666251 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:14.666625 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:14.666628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:14.666637 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:12:14.666779 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:14.666786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.667732 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.667736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:14.668757 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:14.668767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.669275 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.669279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:14.669374 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.669405 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:12:14.671663 (+ 2258us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 06:12:14.673768 (+ 2105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.133.190:44959: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44089/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":1886,"thread_start_us":157,"threads_started":1}
I20260807 06:12:14.674683 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 17223
E20260807 06:12:14.687988 17358 webserver.cc:533] Webserver: sslize failed:
E20260807 06:12:14.688138 17358 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T06:12:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (401 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T06:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:14Z Disabled control of system clock
I20260807 06:12:14.737082 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39917
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:39917
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/testchainca.cert with env {}
W20260807 06:12:14.877537 17363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:14.877930 17363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:14.877983 17363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:14.882547 17363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:14.882647 17363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:14.882663 17363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:14.882681 17363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:14.882697 17363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 06:12:14.882714 17363 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 06:12:14.882728 17363 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 06:12:14.888700 17363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:39917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:14.890022 17363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:14.891279 17363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:14.901249 17374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:14.901264 17375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:14.901674 17363 server_base.cc:1034] running on GCE node
W20260807 06:12:14.901264 17377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:14.902407 17363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:14.903870 17363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:14.905097 17363 hybrid_clock.cc:648] HybridClock initialized: now 1786083134905067 us; error 47 us; skew 500 ppm
I20260807 06:12:14.907410 17363 webserver.cc:460] Webserver started at http://127.9.133.190:44355/ using document root <none> and password file <none>
I20260807 06:12:14.908131 17363 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:14.908236 17363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:14.908514 17363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:14.911049 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "dded8ff5d1754ce1bf7a942696eb3a82"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "113dfa2c91d51472961da93d588d6bdb"
server_key_iv: "e79614536e979c123f74017f0b079461"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.911737 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "dded8ff5d1754ce1bf7a942696eb3a82"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "113dfa2c91d51472961da93d588d6bdb"
server_key_iv: "e79614536e979c123f74017f0b079461"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.916174 17363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:14.919207 17385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:14.920770 17363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:12:14.920959 17363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "dded8ff5d1754ce1bf7a942696eb3a82"
format_stamp: "Formatted at 2026-08-07 06:12:14 on dist-test-slave-h3ft"
server_key: "113dfa2c91d51472961da93d588d6bdb"
server_key_iv: "e79614536e979c123f74017f0b079461"
server_key_version: "encryptionkey@0"
I20260807 06:12:14.921110 17363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:14.944095 17363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:14.945103 17363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:14.945358 17363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:14.954170 17363 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:39917
I20260807 06:12:14.954201 17448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:39917 every 8 connection(s)
I20260807 06:12:14.955451 17363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:14.959509 17450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:14.965734 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 17363
I20260807 06:12:14.965907 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:14.966254 9750 external_mini_cluster.cc:1428] Setting key 3b17d006bbff3e58bc37831772a741f1
I20260807 06:12:14.966848 17450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82: Bootstrap starting.
I20260807 06:12:14.970204 17450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:14.971366 17450 log.cc:824] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:14.974001 17450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82: No bootstrap required, opened a new log
I20260807 06:12:14.978116 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:14.968103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49496 (local address 127.9.133.190:39917)
0807 06:12:14.968974 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:14.968997 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:14.969050 (+ 53us) server_negotiation.cc:407] Connection header received
0807 06:12:14.970010 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:14.970044 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:14.970686 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:14.971191 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:14.971722 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.973213 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.974223 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:14.974757 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:14.974945 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:14.976030 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:14.976036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:14.976053 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:14.976522 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:14.976598 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:14.976620 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:14.976740 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:14.976902 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 06:12:14.977211 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":144,"threads_started":1}
I20260807 06:12:14.978556 17450 raft_consensus.cc:348] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39917 } }
I20260807 06:12:14.978855 17450 raft_consensus.cc:374] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:14.978945 17450 raft_consensus.cc:729] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dded8ff5d1754ce1bf7a942696eb3a82, State: Initialized, Role: FOLLOWER
I20260807 06:12:14.979558 17450 consensus_queue.cc:260] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39917 } }
I20260807 06:12:14.979745 17450 raft_consensus.cc:388] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:14.979843 17450 raft_consensus.cc:482] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:14.979962 17450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:14.981300 17450 raft_consensus.cc:504] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39917 } }
I20260807 06:12:14.981789 17450 leader_election.cc:302] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dded8ff5d1754ce1bf7a942696eb3a82; no voters:
I20260807 06:12:14.981776 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:12:14.982457 17450 leader_election.cc:288] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:14.982543 17457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:14.982831 17457 raft_consensus.cc:686] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [term 1 LEADER]: Becoming Leader. State: Replica: dded8ff5d1754ce1bf7a942696eb3a82, State: Running, Role: LEADER
I20260807 06:12:14.983449 17457 consensus_queue.cc:237] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39917 } }
I20260807 06:12:14.984265 17450 sys_catalog.cc:564] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:14.985771 17458 sys_catalog.cc:455] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dded8ff5d1754ce1bf7a942696eb3a82. Latest consensus state: current_term: 1 leader_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39917 } } }
I20260807 06:12:14.985796 17459 sys_catalog.cc:455] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dded8ff5d1754ce1bf7a942696eb3a82" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39917 } } }
I20260807 06:12:14.985968 17458 sys_catalog.cc:458] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:14.985968 17459 sys_catalog.cc:458] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:14.986416 17470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:14.990235 17470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:14.997141 17470 catalog_manager.cc:1347] Generated new cluster ID: 254e4f31883247e295014982e62ebd36
I20260807 06:12:14.997244 17470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:15.016844 17470 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:15.018301 17470 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:15.027092 17470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dded8ff5d1754ce1bf7a942696eb3a82: Generated new TSK 0
I20260807 06:12:15.028101 17470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:15.510264 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:15.044032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.9.133.190:39917)
0807 06:12:15.044224 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:15.044230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:15.044464 (+ 234us) server_negotiation.cc:407] Connection header received
0807 06:12:15.044695 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:15.044698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:15.044759 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:15.044863 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0807 06:12:15.045160 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:15.045783 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:15.047128 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:15.047385 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:15.047483 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:15.047860 (+ 377us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 06:12:15.509671 (+461811us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 06:12:15.510090 (+ 419us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49508: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 06:12:15.510372 17455 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49508: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 06:12:15.510620 17493 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:15.043926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:39917 (local address 127.0.0.1:49508)
0807 06:12:15.044334 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:15.044351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:15.044511 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:15.044886 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:15.044891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:15.044900 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 06:12:15.045040 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:15.045047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:15.045902 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:15.045907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:15.046949 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:15.046960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:15.047524 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:15.047528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:15.047630 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:15.047663 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 06:12:15.510012 (+462349us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 06:12:15.510331 (+ 319us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.133.190:39917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":177,"threads_started":1}
I20260807 06:12:15.511473 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 17363
2026-08-07T06:12:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (819 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (156 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (157 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:15 dist-test-slave-h3ft krb5kdc[17512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:15 dist-test-slave-h3ft krb5kdc[17512](info): set up 2 sockets
Aug 07 06:12:15 dist-test-slave-h3ft krb5kdc[17512](info): commencing operation
krb5kdc: starting...
W20260807 06:12:17.119148 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:17 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:17.275285 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35379
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:35379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:17.416734 17529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:17.417150 17529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:17.417200 17529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:17.421676 17529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:17.421764 17529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:17.421787 17529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:17.421804 17529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:17.421819 17529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:17.428238 17529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:35379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:17.429864 17529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:17.431321 17529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:17.442135 17538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:17.442135 17537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:17.442132 17540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:17.442587 17529 server_base.cc:1034] running on GCE node
I20260807 06:12:17.443409 17529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:17.444895 17529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:17.446110 17529 hybrid_clock.cc:648] HybridClock initialized: now 1786083137446089 us; error 37 us; skew 500 ppm
Aug 07 06:12:17 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:17.449548 17529 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:17.450700 17529 webserver.cc:460] Webserver started at http://127.9.133.190:43363/ using document root <none> and password file <none>
I20260807 06:12:17.451478 17529 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:17.451582 17529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:17.451871 17529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:17.454517 17529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "80421fa4efd94be5b18632b4e6d56796"
format_stamp: "Formatted at 2026-08-07 06:12:17 on dist-test-slave-h3ft"
server_key: "ea2a6f62e05ea0c1549bf83b0650214a"
server_key_iv: "10981791e017398952b01a45382fefc2"
server_key_version: "encryptionkey@0"
I20260807 06:12:17.455309 17529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "80421fa4efd94be5b18632b4e6d56796"
format_stamp: "Formatted at 2026-08-07 06:12:17 on dist-test-slave-h3ft"
server_key: "ea2a6f62e05ea0c1549bf83b0650214a"
server_key_iv: "10981791e017398952b01a45382fefc2"
server_key_version: "encryptionkey@0"
I20260807 06:12:17.460506 17529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 06:12:17.463945 17548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:17.465682 17529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:12:17.465935 17529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "80421fa4efd94be5b18632b4e6d56796"
format_stamp: "Formatted at 2026-08-07 06:12:17 on dist-test-slave-h3ft"
server_key: "ea2a6f62e05ea0c1549bf83b0650214a"
server_key_iv: "10981791e017398952b01a45382fefc2"
server_key_version: "encryptionkey@0"
I20260807 06:12:17.466130 17529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:17.485954 17529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:17.489775 17529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:17.490060 17529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:17.499038 17529 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:35379
I20260807 06:12:17.499117 17620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:35379 every 8 connection(s)
I20260807 06:12:17.500360 17529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:17.504136 17621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:17.505592 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 17529
I20260807 06:12:17.505754 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:17.506168 9750 external_mini_cluster.cc:1428] Setting key c0004548ca748aeb7eb1d2112c7a0b60
I20260807 06:12:17.512467 17621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796: Bootstrap starting.
I20260807 06:12:17.515995 17621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796: Neither blocks nor log segments found. Creating new log.
Aug 07 06:12:17 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:17.517052 17621 log.cc:824] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:17.520016 17621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796: No bootstrap required, opened a new log
I20260807 06:12:17.523769 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:17.508028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.9.133.190:35379)
0807 06:12:17.508545 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:17.508565 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:17.508610 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:12:17.509344 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:17.509366 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:17.509774 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:17.510120 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:17.511254 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:17.512427 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:17.513613 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:17.514189 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:17.514343 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:17.517765 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:17.517771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:17.517787 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:17.517815 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:17.519901 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:17.520459 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:17.520469 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:17.520479 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:17.520651 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:17.520926 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:17.520929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:17.520931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:17.521527 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:17.521763 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:17.522007 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 06:12:17.522383 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":166,"threads_started":1}
I20260807 06:12:17.525511 17621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80421fa4efd94be5b18632b4e6d56796" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35379 } }
I20260807 06:12:17.525784 17621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:17.525853 17621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80421fa4efd94be5b18632b4e6d56796, State: Initialized, Role: FOLLOWER
I20260807 06:12:17.526460 17621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80421fa4efd94be5b18632b4e6d56796" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35379 } }
I20260807 06:12:17.526647 17621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:17.526726 17621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:17.526837 17621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:17.528059 17621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80421fa4efd94be5b18632b4e6d56796" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35379 } }
I20260807 06:12:17.528486 17621 leader_election.cc:302] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80421fa4efd94be5b18632b4e6d56796; no voters:
I20260807 06:12:17.528867 17621 leader_election.cc:288] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:17.529031 17628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:17.529309 17628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [term 1 LEADER]: Becoming Leader. State: Replica: 80421fa4efd94be5b18632b4e6d56796, State: Running, Role: LEADER
I20260807 06:12:17.529695 17628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80421fa4efd94be5b18632b4e6d56796" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35379 } }
I20260807 06:12:17.530378 17621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:17.531839 17630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80421fa4efd94be5b18632b4e6d56796. Latest consensus state: current_term: 1 leader_uuid: "80421fa4efd94be5b18632b4e6d56796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80421fa4efd94be5b18632b4e6d56796" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35379 } } }
I20260807 06:12:17.531996 17630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:17.532361 17629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80421fa4efd94be5b18632b4e6d56796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80421fa4efd94be5b18632b4e6d56796" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35379 } } }
I20260807 06:12:17.532562 17629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:17.532903 17638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:17.536844 17638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:17.544148 17638 catalog_manager.cc:1347] Generated new cluster ID: 071c67748d8c436797263b6cf409ed44
I20260807 06:12:17.544267 17638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:17.559938 17638 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:17.561529 17638 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:17.580766 17638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80421fa4efd94be5b18632b4e6d56796: Generated new TSK 0
I20260807 06:12:17.581871 17638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:17.632787 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35379
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:17.797252 17653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:17.797675 17653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:17.797755 17653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:17.802275 17653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:17.802380 17653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:17.802451 17653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:17.808580 17653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:17.810168 17653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:17.811653 17653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:17.823616 17662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:17.823632 17665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:17.823632 17661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:17.824137 17653 server_base.cc:1034] running on GCE node
I20260807 06:12:17.824663 17653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:17.825457 17653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:17.826679 17653 hybrid_clock.cc:648] HybridClock initialized: now 1786083137826659 us; error 29 us; skew 500 ppm
Aug 07 06:12:17 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:17.830325 17653 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:17.831880 17653 webserver.cc:460] Webserver started at http://127.9.133.129:41517/ using document root <none> and password file <none>
I20260807 06:12:17.832805 17653 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:17.832926 17653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:17.833235 17653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:17.836226 17653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "6847a949baa6494f9b7b9446e3759903"
format_stamp: "Formatted at 2026-08-07 06:12:17 on dist-test-slave-h3ft"
server_key: "29a9ad0d0a666dc0a4731c3e3608c206"
server_key_iv: "dde1aa889e462ac6a99ff1c363e211c4"
server_key_version: "encryptionkey@0"
I20260807 06:12:17.837016 17653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "6847a949baa6494f9b7b9446e3759903"
format_stamp: "Formatted at 2026-08-07 06:12:17 on dist-test-slave-h3ft"
server_key: "29a9ad0d0a666dc0a4731c3e3608c206"
server_key_iv: "dde1aa889e462ac6a99ff1c363e211c4"
server_key_version: "encryptionkey@0"
I20260807 06:12:17.842330 17653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 06:12:17.845443 17681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:17.847108 17653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:17.847270 17653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "6847a949baa6494f9b7b9446e3759903"
format_stamp: "Formatted at 2026-08-07 06:12:17 on dist-test-slave-h3ft"
server_key: "29a9ad0d0a666dc0a4731c3e3608c206"
server_key_iv: "dde1aa889e462ac6a99ff1c363e211c4"
server_key_version: "encryptionkey@0"
I20260807 06:12:17.847445 17653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:17.872735 17653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:17.877100 17653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:17.877372 17653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:17.878202 17653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:17.879365 17653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:17.879446 17653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:17.879531 17653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:17.879586 17653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:17.891274 17653 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:38257
I20260807 06:12:17.891301 17831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:38257 every 8 connection(s)
I20260807 06:12:17.892525 17653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:17.894452 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 17653
I20260807 06:12:17.894580 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:17.895104 9750 external_mini_cluster.cc:1428] Setting key 03838727204c47ea8e5936141c22e82c
Aug 07 06:12:17 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:17.910949 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:17.895129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:55127 (local address 127.9.133.190:35379)
0807 06:12:17.895320 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:17.895325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:17.896591 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 06:12:17.898397 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:17.898403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:17.898480 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:17.898597 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:17.900182 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:17.900788 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:17.902088 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:17.902364 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:17.902492 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:17.905920 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:17.905927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:17.905933 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:17.905976 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:17.908088 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:17.908741 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:17.908744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:17.908746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:17.908844 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:17.909441 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:17.909446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:17.909448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:17.909699 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:17.909786 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:17.910627 (+ 841us) server_negotiation.cc:299] Negotiation successful
0807 06:12:17.910782 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:17.912247 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:17.895744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35379 (local address 127.9.133.129:55127)
0807 06:12:17.896394 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:17.896474 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:17.898132 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:17.898802 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:17.898811 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:17.899357 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:17.900000 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:17.900017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:17.900929 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:17.900933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:17.901944 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:17.901955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:17.902556 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:17.902561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:17.902781 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:17.903606 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:17.903627 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:17.905682 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:17.908241 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:17.908249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:17.908268 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:17.908617 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:17.908959 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:17.908962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:17.908963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:17.909313 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:17.909832 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:17.909840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:17.910257 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 06:12:17.910534 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 06:12:17.910965 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":183,"threads_started":1}
I20260807 06:12:17.914088 17833 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35379
I20260807 06:12:17.914440 17833 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:17.915179 17833 heartbeater.cc:511] Master 127.9.133.190:35379 requested a full tablet report, sending...
I20260807 06:12:17.917164 17573 ts_manager.cc:194] Registered new tserver with Master: 6847a949baa6494f9b7b9446e3759903 (127.9.133.129:38257)
I20260807 06:12:17.918603 17573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:55127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:17.930836 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35379
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:18.068770 17842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:18.069142 17842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:18.069242 17842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:18.073870 17842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:18.073966 17842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:18.074033 17842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:18.080210 17842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:18.081823 17842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:18.083292 17842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:18.095798 17855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:18.096468 17850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:18.096407 17842 server_base.cc:1034] running on GCE node
W20260807 06:12:18.096735 17853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:18.097304 17842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:18.098208 17842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:18.099399 17842 hybrid_clock.cc:648] HybridClock initialized: now 1786083138099380 us; error 37 us; skew 500 ppm
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.102882 17842 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:18.104151 17842 webserver.cc:460] Webserver started at http://127.9.133.130:35551/ using document root <none> and password file <none>
I20260807 06:12:18.104874 17842 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:18.104943 17842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:18.105165 17842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:18.107923 17842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "f80adff628004219b63b9865c14642a9"
format_stamp: "Formatted at 2026-08-07 06:12:18 on dist-test-slave-h3ft"
server_key: "0f6bd175380a98a1bf23c57ec4950a14"
server_key_iv: "b803ee8c3c0882d7c01ccf5718a9e84d"
server_key_version: "encryptionkey@0"
I20260807 06:12:18.108603 17842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "f80adff628004219b63b9865c14642a9"
format_stamp: "Formatted at 2026-08-07 06:12:18 on dist-test-slave-h3ft"
server_key: "0f6bd175380a98a1bf23c57ec4950a14"
server_key_iv: "b803ee8c3c0882d7c01ccf5718a9e84d"
server_key_version: "encryptionkey@0"
I20260807 06:12:18.113513 17842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:18.116684 17863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:18.118288 17842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:12:18.118448 17842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "f80adff628004219b63b9865c14642a9"
format_stamp: "Formatted at 2026-08-07 06:12:18 on dist-test-slave-h3ft"
server_key: "0f6bd175380a98a1bf23c57ec4950a14"
server_key_iv: "b803ee8c3c0882d7c01ccf5718a9e84d"
server_key_version: "encryptionkey@0"
I20260807 06:12:18.118603 17842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:18.133769 17842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:18.137478 17842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:18.137743 17842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:18.138535 17842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:18.139652 17842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:18.139703 17842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:18.139778 17842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:18.139822 17842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:18.151360 17842 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:46801
I20260807 06:12:18.151419 18008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:46801 every 8 connection(s)
I20260807 06:12:18.152734 17842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:18.161273 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 17842
I20260807 06:12:18.161404 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:18.161772 9750 external_mini_cluster.cc:1428] Setting key 2541fb5f1220b28b9509ef54eebf203e
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:18.169399 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.155079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:59595 (local address 127.9.133.190:35379)
0807 06:12:18.155263 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.155267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.156315 (+ 1048us) server_negotiation.cc:407] Connection header received
0807 06:12:18.157962 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.157966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.158041 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.158159 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.159659 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.160242 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.161597 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.161826 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.161958 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.165061 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:18.165065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:18.165067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:18.165091 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:18.166919 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.167512 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.167518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.167523 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.167649 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.167998 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.168005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.168009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.168284 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:18.168394 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:18.168954 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 06:12:18.169125 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:12:18.170447 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.155576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35379 (local address 127.9.133.130:59595)
0807 06:12:18.156134 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:18.156187 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:18.157707 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:18.158378 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:18.158387 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:18.158822 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:18.159498 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:18.159513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.160384 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.160388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:18.161391 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:18.161405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.162039 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.162044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:18.162269 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.163116 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:18.163135 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:18.164877 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:18.167089 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:18.167094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:18.167108 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:18.167394 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:18.167746 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:18.167749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:18.167750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:18.167892 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:18.168393 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:18.168398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:18.168674 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 06:12:18.168848 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 06:12:18.169154 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":141,"threads_started":1}
I20260807 06:12:18.171881 18010 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35379
I20260807 06:12:18.172211 18010 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:18.173019 18010 heartbeater.cc:511] Master 127.9.133.190:35379 requested a full tablet report, sending...
I20260807 06:12:18.174396 17573 ts_manager.cc:194] Registered new tserver with Master: f80adff628004219b63b9865c14642a9 (127.9.133.130:46801)
I20260807 06:12:18.175138 17573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:59595
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:18.199416 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35379
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:18.333743 18017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:18.334177 18017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:18.334264 18017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:18.338702 18017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:18.338826 18017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:18.338932 18017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:18.345148 18017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:18.346696 18017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:18.348232 18017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:18.360985 18029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:18.360985 18027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:18.361279 18032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:18.361833 18017 server_base.cc:1034] running on GCE node
I20260807 06:12:18.362342 18017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:18.363229 18017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:18.364404 18017 hybrid_clock.cc:648] HybridClock initialized: now 1786083138364387 us; error 34 us; skew 500 ppm
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.367770 18017 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:18.368953 18017 webserver.cc:460] Webserver started at http://127.9.133.131:45655/ using document root <none> and password file <none>
I20260807 06:12:18.369663 18017 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:18.369767 18017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:18.370025 18017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:18.372709 18017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "d2ca30875b7d4e339a6ea8a65e6f5a42"
format_stamp: "Formatted at 2026-08-07 06:12:18 on dist-test-slave-h3ft"
server_key: "24edfaa7d0580f6c78f78eaf29cf4423"
server_key_iv: "e61b8b5adfa8249cc46aeabad33b3588"
server_key_version: "encryptionkey@0"
I20260807 06:12:18.373448 18017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "d2ca30875b7d4e339a6ea8a65e6f5a42"
format_stamp: "Formatted at 2026-08-07 06:12:18 on dist-test-slave-h3ft"
server_key: "24edfaa7d0580f6c78f78eaf29cf4423"
server_key_iv: "e61b8b5adfa8249cc46aeabad33b3588"
server_key_version: "encryptionkey@0"
I20260807 06:12:18.377979 18017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:12:18.381003 18041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:18.382512 18017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:18.382673 18017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "d2ca30875b7d4e339a6ea8a65e6f5a42"
format_stamp: "Formatted at 2026-08-07 06:12:18 on dist-test-slave-h3ft"
server_key: "24edfaa7d0580f6c78f78eaf29cf4423"
server_key_iv: "e61b8b5adfa8249cc46aeabad33b3588"
server_key_version: "encryptionkey@0"
I20260807 06:12:18.382817 18017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:18.406701 18017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:18.410782 18017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:18.411092 18017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:18.411900 18017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:18.413086 18017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:18.413162 18017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:18.413242 18017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:18.413285 18017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:18.424543 18017 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:33273
I20260807 06:12:18.424564 18191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:33273 every 8 connection(s)
I20260807 06:12:18.425822 18017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:18.429565 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 18017
I20260807 06:12:18.429725 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:18.430140 9750 external_mini_cluster.cc:1428] Setting key 0ec7d08dfa72254652dda48503e56e09
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:18.443307 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.428419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:60193 (local address 127.9.133.190:35379)
0807 06:12:18.428601 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.428606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.429739 (+ 1133us) server_negotiation.cc:407] Connection header received
0807 06:12:18.431778 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.431781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.431855 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.431962 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.433663 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.434259 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.435583 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.435809 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.435896 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.439281 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:18.439289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:18.439295 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:18.439341 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:18.440858 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.441480 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.441483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.441484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.441581 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.441964 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.441966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.441967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.442180 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:18.442259 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:18.442849 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 06:12:18.443081 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:12:18.444372 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.428927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35379 (local address 127.9.133.131:60193)
0807 06:12:18.429534 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:18.429591 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:18.431524 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:18.432293 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:18.432303 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:18.432786 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:18.433511 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:18.433529 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.434372 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.434374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:18.435470 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:18.435480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.435952 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.435955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:18.436121 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.436844 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:18.436871 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:18.438988 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:18.440991 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:18.440996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:18.441013 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:18.441363 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:18.441675 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:18.441678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:18.441679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:18.441833 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:18.442270 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:18.442275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:18.442578 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 06:12:18.442758 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 06:12:18.443165 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":199,"threads_started":1}
I20260807 06:12:18.445746 18192 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35379
I20260807 06:12:18.446019 18192 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:18.446600 18192 heartbeater.cc:511] Master 127.9.133.190:35379 requested a full tablet report, sending...
I20260807 06:12:18.447876 17573 ts_manager.cc:194] Registered new tserver with Master: d2ca30875b7d4e339a6ea8a65e6f5a42 (127.9.133.131:33273)
I20260807 06:12:18.448571 17573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:60193
I20260807 06:12:18.457819 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/test-admin.keytab.
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.480880 9750 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 06:12:18.484392 18201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:18.482640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.9.133.129:38257)
0807 06:12:18.483126 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.483135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.483165 (+ 30us) server_negotiation.cc:407] Connection header received
0807 06:12:18.483234 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.483240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.483483 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.483627 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.484179 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44390: BlockingRecv error: recv got EOF from 127.0.0.1:44390 (error 108)
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":185,"threads_started":1}
W20260807 06:12:18.484836 9750 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc' of type FILE
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.486019 9750 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.487167 9750 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:12:18.497853 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.488310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.9.133.129:38257)
0807 06:12:18.488436 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.488440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.488645 (+ 205us) server_negotiation.cc:407] Connection header received
0807 06:12:18.488704 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.488707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.488756 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.488836 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.489390 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.490236 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.491139 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.491686 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.491796 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.494278 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:18.494289 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:18.494306 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:18.494331 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:18.496081 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.496539 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.496546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.496548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.496633 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.496895 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.496897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.496899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.497193 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:18.497383 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:18.497556 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 06:12:18.497703 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 06:12:18.502683 18201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:18.501671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.9.133.129:38257)
0807 06:12:18.501797 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.501801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.501931 (+ 130us) server_negotiation.cc:407] Connection header received
0807 06:12:18.502050 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.502053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.502101 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.502177 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.502563 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44412: BlockingRecv error: recv got EOF from 127.0.0.1:44412 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 06:12:18.503125 9750 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc)
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.504366 9750 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.505388 9750 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:12:18.515411 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.506444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.9.133.129:38257)
0807 06:12:18.506573 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.506577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.506705 (+ 128us) server_negotiation.cc:407] Connection header received
0807 06:12:18.506819 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.506822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.506871 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.506957 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.507559 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.508363 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.509185 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.509549 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.509651 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.512180 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:18.512185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:18.512188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:18.512217 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:18.513841 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.514237 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.514240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.514241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.514319 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.514570 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.514572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.514573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.514752 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:18.514884 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:18.515155 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 06:12:18.515268 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 06:12:18.518570 18201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:18.517546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.9.133.129:38257)
0807 06:12:18.517683 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.517689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.517755 (+ 66us) server_negotiation.cc:407] Connection header received
0807 06:12:18.517915 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.517918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.517966 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.518054 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.518462 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46330: BlockingRecv error: recv got EOF from 127.0.0.1:46330 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 06:12:18.518865 9750 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786083100784614-9750-0/krb5kdc/krb5cc' of type FILE
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.520053 9750 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:18.520994 9750 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[17512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:12:18.530426 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:18.521923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.9.133.129:38257)
0807 06:12:18.522065 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:18.522068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:18.522081 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:12:18.522196 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:18.522198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:18.522234 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:18.522298 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:18.522787 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.523439 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.524352 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:18.524569 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:18.524643 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:18.527131 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:18.527138 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:18.527142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:18.527175 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:18.528621 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.529061 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.529065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.529066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.529163 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:18.529419 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:18.529428 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:18.529430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:18.529628 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:18.529766 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:18.530122 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 06:12:18.530272 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 06:12:18.531957 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 17653
I20260807 06:12:18.543721 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 17842
I20260807 06:12:18.571754 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 18017
I20260807 06:12:18.585820 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 17529
2026-08-07T06:12:18Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2759 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[18217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[18217](info): set up 2 sockets
Aug 07 06:12:18 dist-test-slave-h3ft krb5kdc[18217](info): commencing operation
krb5kdc: starting...
W20260807 06:12:19.850687 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:19 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab.
Entry for principal oryx/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab.
I20260807 06:12:20.006718 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:34569
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab
--principal=oryx/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:34569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:20.148442 18232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:20.148813 18232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:20.148859 18232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:20.153149 18232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:20.153223 18232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:20.153244 18232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:20.153261 18232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:20.153277 18232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:20.159106 18232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:34569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab
--principal=oryx/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:34569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:20.160581 18232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:20.162015 18232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:20.173045 18238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:20.173107 18237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:20.173063 18241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:20.173534 18232 server_base.cc:1034] running on GCE node
I20260807 06:12:20.174501 18232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:20.176039 18232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:20.177270 18232 hybrid_clock.cc:648] HybridClock initialized: now 1786083140177248 us; error 37 us; skew 500 ppm
Aug 07 06:12:20 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:20.186015 18232 init.cc:377] Logged in from keytab as oryx/127.9.133.190@KRBTEST.COM (short username oryx)
I20260807 06:12:20.187529 18232 webserver.cc:460] Webserver started at http://127.9.133.190:45525/ using document root <none> and password file <none>
I20260807 06:12:20.188320 18232 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:20.188444 18232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:20.188762 18232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:20.192195 18232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "8896cbe9e2a74deb83262b48f4d99d2d"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "0aed5190d0de20ec82ffccc45d5252f4"
server_key_iv: "e1c1de441a9420fc3676f86a6ca1f1fd"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.192977 18232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "8896cbe9e2a74deb83262b48f4d99d2d"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "0aed5190d0de20ec82ffccc45d5252f4"
server_key_iv: "e1c1de441a9420fc3676f86a6ca1f1fd"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.197957 18232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:12:20.201233 18255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.202850 18232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:20.203055 18232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "8896cbe9e2a74deb83262b48f4d99d2d"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "0aed5190d0de20ec82ffccc45d5252f4"
server_key_iv: "e1c1de441a9420fc3676f86a6ca1f1fd"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.203200 18232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:20.220556 18232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:20.224334 18232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:20.224597 18232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:20.233618 18232 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:34569
I20260807 06:12:20.234140 18333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:34569 every 8 connection(s)
I20260807 06:12:20.234866 18232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:20.237921 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 18232
I20260807 06:12:20.238086 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:20.238555 9750 external_mini_cluster.cc:1428] Setting key 20c77bbafaf40ac6a8d5e6ee777878de
I20260807 06:12:20.238952 18334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:20.247659 18334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Bootstrap starting.
Aug 07 06:12:20 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:20.250990 18334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:20.251951 18334 log.cc:824] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:20.254408 18334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: No bootstrap required, opened a new log
I20260807 06:12:20.255332 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:20.240289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.9.133.190:34569)
0807 06:12:20.240786 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:20.240804 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:20.240847 (+ 43us) server_negotiation.cc:407] Connection header received
0807 06:12:20.241634 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:20.241653 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:20.242042 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:20.242370 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:20.243552 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.244666 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.245903 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.246440 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.246579 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:20.249886 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:20.249891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:20.249908 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:20.249942 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:20.251955 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:20.252460 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:20.252471 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:20.252476 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:20.252619 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:20.252932 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:20.252936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:20.252937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:20.253417 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:20.253667 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:20.253972 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 06:12:20.254318 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":175,"threads_started":1}
I20260807 06:12:20.259116 18334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } }
I20260807 06:12:20.259413 18334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:20.259500 18334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8896cbe9e2a74deb83262b48f4d99d2d, State: Initialized, Role: FOLLOWER
I20260807 06:12:20.260051 18334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } }
I20260807 06:12:20.260219 18334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:20.260313 18334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:20.260432 18334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:20.261636 18334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } }
I20260807 06:12:20.262074 18334 leader_election.cc:302] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8896cbe9e2a74deb83262b48f4d99d2d; no voters:
I20260807 06:12:20.262487 18334 leader_election.cc:288] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:20.262622 18343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:20.262887 18343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 LEADER]: Becoming Leader. State: Replica: 8896cbe9e2a74deb83262b48f4d99d2d, State: Running, Role: LEADER
I20260807 06:12:20.263342 18343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } }
I20260807 06:12:20.264025 18334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:20.265436 18344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } }
I20260807 06:12:20.265622 18344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:20.265436 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8896cbe9e2a74deb83262b48f4d99d2d. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } }
I20260807 06:12:20.265689 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
W20260807 06:12:20.269802 18365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:12:20.269912 18365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:12:20.270007 18356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:20.271142 18356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:20.278129 18356 catalog_manager.cc:1347] Generated new cluster ID: 30ab31c3798842e1b005d70a8070818e
I20260807 06:12:20.278234 18356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:20.292520 18356 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:20.293952 18356 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:20.309077 18356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Generated new TSK 0
I20260807 06:12:20.310343 18356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab.
Entry for principal oryx/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab.
I20260807 06:12:20.351473 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34569
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:20.494589 18374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:20.495078 18374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:20.495187 18374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:20.499580 18374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:20.499668 18374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:20.499737 18374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:20.505980 18374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:20.507674 18374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:20.509137 18374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:20.521603 18386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:20.521620 18383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:20.521603 18382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:20.522106 18374 server_base.cc:1034] running on GCE node
I20260807 06:12:20.522680 18374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:20.523691 18374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:20.524914 18374 hybrid_clock.cc:648] HybridClock initialized: now 1786083140524892 us; error 52 us; skew 500 ppm
Aug 07 06:12:20 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:20.528678 18374 init.cc:377] Logged in from keytab as oryx/127.9.133.129@KRBTEST.COM (short username oryx)
I20260807 06:12:20.529847 18374 webserver.cc:460] Webserver started at http://127.9.133.129:35365/ using document root <none> and password file <none>
I20260807 06:12:20.530622 18374 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:20.530727 18374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:20.531090 18374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:20.533705 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "0c8e26a4d040484fbb0967275acfa7ab"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "d4380fd78905b4ddf91e91bd11a60cd8"
server_key_iv: "5f0b0567af25e186f3aa6b854e421146"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.534404 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "0c8e26a4d040484fbb0967275acfa7ab"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "d4380fd78905b4ddf91e91bd11a60cd8"
server_key_iv: "5f0b0567af25e186f3aa6b854e421146"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.539403 18374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 06:12:20.542470 18397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.544178 18374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:20.544339 18374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "0c8e26a4d040484fbb0967275acfa7ab"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "d4380fd78905b4ddf91e91bd11a60cd8"
server_key_iv: "5f0b0567af25e186f3aa6b854e421146"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.544480 18374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:20.555953 18374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:20.559971 18374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:20.560228 18374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:20.560976 18374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:20.561990 18374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:20.562067 18374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.562150 18374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:20.562206 18374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.573936 18374 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:35435
I20260807 06:12:20.573969 18553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:35435 every 8 connection(s)
I20260807 06:12:20.575201 18374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:20.581947 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 18374
I20260807 06:12:20.582135 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:20.582571 9750 external_mini_cluster.cc:1428] Setting key fe1225fda32f9ef7d334bb973b8c26f2
Aug 07 06:12:20 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:20.594053 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:20.577687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:41441 (local address 127.9.133.190:34569)
0807 06:12:20.577910 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:20.577915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:20.578969 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 06:12:20.580716 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:20.580720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:20.580787 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:20.580878 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:20.582521 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.583383 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.584822 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.585092 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.585180 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:20.589024 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:20.589030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:20.589040 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:20.589077 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:20.591141 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:20.591987 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:20.591992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:20.591994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:20.592115 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:20.592511 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:20.592518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:20.592523 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:20.592854 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:20.592981 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:20.593611 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 06:12:20.593802 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 06:12:20.595167 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:20.578201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:41441)
0807 06:12:20.578777 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:20.578830 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:20.580448 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:20.581090 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:20.581097 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:20.581580 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:20.582321 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:20.582340 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.583544 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.583548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:20.584692 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:20.584705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.585273 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.585280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:20.585546 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:20.586343 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:20.586382 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:20.588787 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:20.591303 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:20.591313 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:20.591330 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:20.591863 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:20.592225 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:20.592227 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:20.592229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:20.592400 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:20.592983 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:20.592989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:20.593316 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 06:12:20.593522 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 06:12:20.593875 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":141,"threads_started":1}
WARNING: no policy specified for oryx/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:20.596954 18554 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34569
I20260807 06:12:20.597273 18554 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:20.597903 18554 heartbeater.cc:511] Master 127.9.133.190:34569 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:20.599982 18288 ts_manager.cc:194] Registered new tserver with Master: 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435)
I20260807 06:12:20.601521 18288 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:41441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab.
Entry for principal oryx/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab.
I20260807 06:12:20.622148 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34569
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:20.773646 18563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:20.773991 18563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:20.774045 18563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:20.778730 18563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:20.778807 18563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:20.778878 18563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:20.784910 18563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:20.786337 18563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:20.787678 18563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:20.799458 18569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:20.799481 18573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:20.799484 18570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:20.800069 18563 server_base.cc:1034] running on GCE node
I20260807 06:12:20.800613 18563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:20.801467 18563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:20.802680 18563 hybrid_clock.cc:648] HybridClock initialized: now 1786083140802650 us; error 43 us; skew 500 ppm
Aug 07 06:12:20 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:20.806237 18563 init.cc:377] Logged in from keytab as oryx/127.9.133.130@KRBTEST.COM (short username oryx)
I20260807 06:12:20.807590 18563 webserver.cc:460] Webserver started at http://127.9.133.130:38897/ using document root <none> and password file <none>
I20260807 06:12:20.808449 18563 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:20.808562 18563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:20.808869 18563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:20.811704 18563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "fb6649c6dea548c98d4bedc1b73d852e"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "68effc671786a64bba8991a6b7dcb16f"
server_key_iv: "8e8ce919935a650c9fded8bce4c0acbe"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.812434 18563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "fb6649c6dea548c98d4bedc1b73d852e"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "68effc671786a64bba8991a6b7dcb16f"
server_key_iv: "8e8ce919935a650c9fded8bce4c0acbe"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.817366 18563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:20.820415 18583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.821942 18563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:12:20.822103 18563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "fb6649c6dea548c98d4bedc1b73d852e"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "68effc671786a64bba8991a6b7dcb16f"
server_key_iv: "8e8ce919935a650c9fded8bce4c0acbe"
server_key_version: "encryptionkey@0"
I20260807 06:12:20.822237 18563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:20.847294 18563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:20.853200 18563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:20.853433 18563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:20.854173 18563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:20.855301 18563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:20.855362 18563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.855415 18563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:20.855433 18563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:20.866730 18563 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:46105
I20260807 06:12:20.866828 18738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:46105 every 8 connection(s)
I20260807 06:12:20.867951 18563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:20.873227 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 18563
I20260807 06:12:20.873433 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:20.873809 9750 external_mini_cluster.cc:1428] Setting key 42c5d64d3dac8c6190a3bb8c9df69b45
Aug 07 06:12:20 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:20.884914 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:20.870475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:36429 (local address 127.9.133.190:34569)
0807 06:12:20.870676 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:20.870682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:20.871735 (+ 1053us) server_negotiation.cc:407] Connection header received
0807 06:12:20.873292 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:20.873296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:20.873370 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:20.873484 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:20.875123 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.875812 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.877107 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.877336 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.877453 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:20.880760 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:20.880764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:20.880766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:20.880794 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:20.882450 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:20.883084 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:20.883088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:20.883089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:20.883178 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:20.883573 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:20.883576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:20.883578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:20.883800 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:20.883914 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:20.884555 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 06:12:20.884711 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 06:12:20.886029 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:20.870967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:36429)
0807 06:12:20.871556 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:20.871608 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:20.873042 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:20.873709 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:20.873717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:20.874240 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:20.874921 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:20.874938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.875960 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.875963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:20.876943 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:20.876953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:20.877506 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:20.877510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:20.877711 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:20.878444 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:20.878465 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:20.880477 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:20.882592 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:20.882599 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:20.882620 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:20.882930 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:20.883278 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:20.883282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:20.883285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:20.883460 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:20.883929 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:20.883939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:20.884268 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 06:12:20.884491 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 06:12:20.884922 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for oryx/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:12:20.887540 18739 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34569
I20260807 06:12:20.887897 18739 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:20.888553 18739 heartbeater.cc:511] Master 127.9.133.190:34569 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.131@KRBTEST.COM" created.
I20260807 06:12:20.889983 18288 ts_manager.cc:194] Registered new tserver with Master: fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:20.890825 18288 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:36429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab.
Entry for principal oryx/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab.
I20260807 06:12:20.910878 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34569
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:21.057106 18744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:21.057511 18744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:21.057592 18744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:21.061966 18744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:21.062091 18744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:21.062193 18744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:21.068701 18744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:34569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:21.070377 18744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:21.071869 18744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:21.084467 18753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.084501 18755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.084481 18751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:21.084933 18744 server_base.cc:1034] running on GCE node
I20260807 06:12:21.085430 18744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:21.086319 18744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:21.087499 18744 hybrid_clock.cc:648] HybridClock initialized: now 1786083141087480 us; error 35 us; skew 500 ppm
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:21.090860 18744 init.cc:377] Logged in from keytab as oryx/127.9.133.131@KRBTEST.COM (short username oryx)
I20260807 06:12:21.092007 18744 webserver.cc:460] Webserver started at http://127.9.133.131:39749/ using document root <none> and password file <none>
I20260807 06:12:21.092708 18744 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:21.092777 18744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:21.092996 18744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:21.095660 18744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "a95e253980cf47a1ab96894a38a3d371"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "03535c9cbdf38dd86caca5c489716a28"
server_key_iv: "977573786064f5e6ab3ba3b1418b7797"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.096421 18744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "a95e253980cf47a1ab96894a38a3d371"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "03535c9cbdf38dd86caca5c489716a28"
server_key_iv: "977573786064f5e6ab3ba3b1418b7797"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.101161 18744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 06:12:21.104385 18769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.106076 18744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:21.106249 18744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "a95e253980cf47a1ab96894a38a3d371"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "03535c9cbdf38dd86caca5c489716a28"
server_key_iv: "977573786064f5e6ab3ba3b1418b7797"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.106395 18744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:21.164724 18744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:21.169039 18744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:21.169298 18744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:21.170118 18744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:21.171391 18744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:21.171447 18744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.171525 18744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:21.171586 18744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.182435 18744 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:39479
I20260807 06:12:21.182456 18918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:39479 every 8 connection(s)
I20260807 06:12:21.183655 18744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:21.192229 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 18744
I20260807 06:12:21.192425 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:21.192795 9750 external_mini_cluster.cc:1428] Setting key 297976b697d9a7f246868feea35b4002
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:21.200486 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.186016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:56361 (local address 127.9.133.190:34569)
0807 06:12:21.186200 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.186204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.187204 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 06:12:21.188792 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.188816 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.188893 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.189008 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.190418 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.191052 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.192314 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.192682 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.192801 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.196176 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.196180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.196182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.196210 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.197975 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.198568 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.198572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.198573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.198671 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.199133 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.199136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.199137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.199369 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.199461 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.200172 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.200310 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:12:21.201534 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.186452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:56361)
0807 06:12:21.186977 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.187085 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.188538 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.189222 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.189230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.189664 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.190248 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.190263 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.191189 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.191199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.192182 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.192191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.192850 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.192855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.193069 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.193807 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.193832 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.195993 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.198107 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.198114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.198130 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.198461 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.198765 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.198768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.198770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.198965 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.199477 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.199484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.199798 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.200040 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.200374 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":133,"threads_started":1}
I20260807 06:12:21.203076 18920 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34569
I20260807 06:12:21.203413 18920 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:21.204082 18920 heartbeater.cc:511] Master 127.9.133.190:34569 requested a full tablet report, sending...
I20260807 06:12:21.205403 18288 ts_manager.cc:194] Registered new tserver with Master: a95e253980cf47a1ab96894a38a3d371 (127.9.133.131:39479)
I20260807 06:12:21.206158 18288 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:56361
I20260807 06:12:21.207937 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.133.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.189.keytab.
Entry for principal oryx/127.9.133.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.189.keytab.
I20260807 06:12:21.246477 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:38265
--webserver_interface=127.9.133.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.189.keytab
--principal=oryx/127.9.133.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:34569,127.9.133.189:38265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:21.385481 18926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:21.385819 18926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:21.385864 18926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:21.390151 18926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:21.390232 18926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:21.390252 18926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:21.390271 18926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:21.390287 18926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:21.396171 18926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:34569,127.9.133.189:38265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.189.keytab
--principal=oryx/127.9.133.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:38265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.133.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
W20260807 06:12:21.396464 18926 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.133.190:34569,127.9.133.189:38265'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 06:12:21.397651 18926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:21.398988 18926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:21.409914 18940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.409886 18936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.409886 18935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:21.410132 18926 server_base.cc:1034] running on GCE node
I20260807 06:12:21.411221 18926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:21.412760 18926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:21.413976 18926 hybrid_clock.cc:648] HybridClock initialized: now 1786083141413948 us; error 38 us; skew 500 ppm
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:21.417733 18926 init.cc:377] Logged in from keytab as oryx/127.9.133.189@KRBTEST.COM (short username oryx)
I20260807 06:12:21.418921 18926 webserver.cc:460] Webserver started at http://127.9.133.189:44935/ using document root <none> and password file <none>
I20260807 06:12:21.419667 18926 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:21.419739 18926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:21.419968 18926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:21.422593 18926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/instance:
uuid: "65e2a832f26d4cbc89f10e700713e982"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "dee1bb7c44a1d458de7461584d53915c"
server_key_iv: "28c07b38bf4caa82287a1f784931e8b4"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.423313 18926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal/instance:
uuid: "65e2a832f26d4cbc89f10e700713e982"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "dee1bb7c44a1d458de7461584d53915c"
server_key_iv: "28c07b38bf4caa82287a1f784931e8b4"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.428234 18926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 06:12:21.431483 18950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.433327 18926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 06:12:21.433513 18926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
uuid: "65e2a832f26d4cbc89f10e700713e982"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "dee1bb7c44a1d458de7461584d53915c"
server_key_iv: "28c07b38bf4caa82287a1f784931e8b4"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.433665 18926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:21.449263 18926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:21.453310 18926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:21.453577 18926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:21.462090 18926 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.189:38265
I20260807 06:12:21.462160 19032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.189:38265 every 8 connection(s)
I20260807 06:12:21.463336 18926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
I20260807 06:12:21.466517 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 18926
I20260807 06:12:21.467599 19033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:21.473652 19033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.189@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:21.485807 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.468749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.9.133.189:38265)
0807 06:12:21.469255 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.469278 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.469329 (+ 51us) server_negotiation.cc:407] Connection header received
0807 06:12:21.470342 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.470373 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.470886 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.471369 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.472329 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.473942 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.475050 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.475722 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.475894 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.479258 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.479265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.479279 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.479313 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.481713 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.482308 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.482315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.482322 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.482463 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.482930 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.482933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.482935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.483567 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.483885 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.484311 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.484675 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":141,"threads_started":1}
I20260807 06:12:21.487434 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.476053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49098)
0807 06:12:21.476575 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.476633 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.476824 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.477215 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.477220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.477627 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.477772 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.477777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.478645 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.478651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.479745 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.479756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.480235 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.480239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.480372 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.480866 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.480885 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.483069 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.485317 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.485329 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.485337 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.485603 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.485977 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.485981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.485983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.486098 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.486716 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.486722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.486833 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.486979 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.487221 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":225,"threads_started":1}
I20260807 06:12:21.487509 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.476117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.9.133.190:34569)
0807 06:12:21.476281 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.476305 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.476726 (+ 421us) server_negotiation.cc:407] Connection header received
0807 06:12:21.476975 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.476979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.477054 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.477159 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.477923 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.478499 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.479869 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.480114 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.480231 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.483250 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.483255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.483265 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.483296 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.485188 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.485729 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.485733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.485735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.485842 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.486218 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.486223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.486225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.486598 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.486718 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.487102 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.487271 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:12:21.489991 19033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.189@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
I20260807 06:12:21.499614 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.490679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60766 (local address 127.9.133.189:38265)
0807 06:12:21.490792 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.490795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.490807 (+ 12us) server_negotiation.cc:407] Connection header received
0807 06:12:21.490890 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.490892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.490932 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.490997 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.491549 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.492532 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.493428 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.493655 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.493739 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.496114 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.496118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.496120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.496151 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.497656 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.498111 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.498115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.498116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.498210 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.498539 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.498542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.498543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.498762 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.498852 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.499346 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.499466 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:12:21.499614 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.490556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60766)
0807 06:12:21.490721 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.490736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.490809 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.491085 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.491087 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.491252 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.491370 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.491391 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.492671 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.492674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.493306 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.493316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.493765 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.493768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.493841 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.494408 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.494421 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.495977 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.497768 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.497770 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.497772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.497992 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.498342 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.498344 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.498346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.498440 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.498928 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.498931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.499109 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.499245 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.499387 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 06:12:21.503719 19033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Bootstrap starting.
I20260807 06:12:21.506785 19033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:21.507750 19033 log.cc:824] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:21.510150 19033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: No bootstrap required, opened a new log
I20260807 06:12:21.513844 19033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } }
I20260807 06:12:21.514103 19033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:21.514142 19033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65e2a832f26d4cbc89f10e700713e982, State: Initialized, Role: FOLLOWER
I20260807 06:12:21.514664 19033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } }
I20260807 06:12:21.515307 19039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } } }
I20260807 06:12:21.515478 19039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:21.516240 19033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 06:12:21.522328 19053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:12:21.522442 19053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:12:21.525220 18926 master_runner.cc:428] Detected that this master 65e2a832f26d4cbc89f10e700713e982 is joining an existing cluster
I20260807 06:12:21.525312 18926 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.133.189:38265
I20260807 06:12:21.538465 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.529111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60778)
0807 06:12:21.529514 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.529529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.529648 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.529937 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.529939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.530228 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.530346 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.530352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.531494 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.531498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.532466 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.532478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.533197 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.533201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.533316 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.534001 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.534017 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.534474 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.536672 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.536676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.536678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.536947 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.537264 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.537267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.537269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.537344 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.537815 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.537818 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.537914 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.538023 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.538172 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":138,"threads_started":1}
I20260807 06:12:21.538870 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.529162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.9.133.189:38265)
0807 06:12:21.529306 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.529311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.529603 (+ 292us) server_negotiation.cc:407] Connection header received
0807 06:12:21.529755 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.529761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.529806 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.529895 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.530469 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.531372 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.532604 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.532901 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.533197 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.534629 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.534633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.534637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.534661 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.536529 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.537069 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.537074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.537076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.537168 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.537437 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.537444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.537448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.537710 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.537824 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.538108 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.538248 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:12:21.540125 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.528596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.9.133.190:34569)
0807 06:12:21.528776 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.528782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.529562 (+ 780us) server_negotiation.cc:407] Connection header received
0807 06:12:21.529736 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.529739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.529811 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.529913 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.530598 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.531440 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.532333 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.532640 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.532743 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.534658 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.534662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.534664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.534697 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.536804 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.537281 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.537287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.537291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.537398 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.537718 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.537721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.537723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.538025 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.538137 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.538525 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.539283 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 06:12:21.540752 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.528477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49100)
0807 06:12:21.529451 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.529469 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.529610 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.530062 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.530065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.530318 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.530455 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.530463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.531531 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.531537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.532210 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.532218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.532800 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.532804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.532898 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.533820 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.533837 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.534485 (+ 648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.536923 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.536926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.536928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.537163 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.537531 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.537534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.537536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.537615 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.538184 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.538187 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.538303 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.540375 (+ 2072us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.540552 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":54,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for oryx/127.9.133.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.188@KRBTEST.COM" created.
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.190@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
I20260807 06:12:21.563159 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.546923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.9.133.189:38265)
0807 06:12:21.547075 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.547079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.547538 (+ 459us) server_negotiation.cc:407] Connection header received
0807 06:12:21.547913 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.547921 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.548010 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.548117 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.549067 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.550098 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.551471 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.551890 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.552031 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.558508 (+ 6477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.558512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.558514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.558543 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.560612 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.561305 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.561312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.561315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.561424 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.561759 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.561762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.561763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.562075 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.562205 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.562725 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.562896 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:12:21.563925 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.546887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60790)
0807 06:12:21.547377 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.547461 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.547650 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.548171 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.548178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.548692 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.548925 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.548935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.550224 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.550228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.551332 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.551345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.552035 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.552039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.552169 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.552896 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.552913 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.558307 (+ 5394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.560775 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.560788 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.560798 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.561159 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.561531 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.561534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.561536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.561646 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.562207 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.562220 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.562401 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.563391 (+ 990us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.563674 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":138,"threads_started":1}
I20260807 06:12:21.565933 18288 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.133.189:38265: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "65e2a832f26d4cbc89f10e700713e982"
member_type: NON_VOTER
last_known_addr {
host: "127.9.133.189"
port: 38265
}
attrs {
promote: true
}
}
dest_uuid: "8896cbe9e2a74deb83262b48f4d99d2d"
cas_config_opid_index: -1
I20260807 06:12:21.566941 18288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } }
I20260807 06:12:21.569828 19070 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:21.571348 18991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:21.572451 19070 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.133.189:38265
I20260807 06:12:21.572726 19070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:21.572829 19070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:21.573107 18926 master.cc:537] Master@127.9.133.189:38265 shutting down...
I20260807 06:12:21.574133 18991 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Refusing update from remote peer 8896cbe9e2a74deb83262b48f4d99d2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 06:12:21.575093 19071 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:12:21.577843 18257 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.188.keytab.
Entry for principal oryx/127.9.133.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.188.keytab.
W20260807 06:12:21.579145 18257 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d -> Peer 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189:38265): Couldn't send request to peer 65e2a832f26d4cbc89f10e700713e982. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 06:12:21.580135 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:37415
--webserver_interface=127.9.133.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.188.keytab
--principal=oryx/127.9.133.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:12:21.585618 18926 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:21.586086 18926 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:21.586305 18926 tablet_replica.cc:330] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: stopping tablet replica
I20260807 06:12:21.593739 18926 master.cc:552] Master@127.9.133.189:38265 shutdown complete.
I20260807 06:12:21.593891 18926 master_runner.cc:305] Clearing existing system tablet
I20260807 06:12:21.594353 18926 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:21.594538 18926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:21.596578 18926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:21.597942 19080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.598191 18926 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:21.598308 18926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:21.598341 18926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
uuid: "65e2a832f26d4cbc89f10e700713e982"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "dee1bb7c44a1d458de7461584d53915c"
server_key_iv: "28c07b38bf4caa82287a1f784931e8b4"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.601161 18926 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:21.604056 18926 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 06:12:21.604166 18926 log.cc:1196] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 06:12:21.604458 18926 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Deleting consensus metadata
I20260807 06:12:21.604660 18926 master_runner.cc:315] Copying system tablet from 127.9.133.190:34569
I20260807 06:12:21.604833 18554 heartbeater.cc:503] Master 127.9.133.190:34569 was elected leader, sending a full tablet report...
I20260807 06:12:21.606258 18926 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: tablet copy: Beginning tablet copy session from remote peer at address 127.9.133.190:34569
I20260807 06:12:21.617719 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.607323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49116)
0807 06:12:21.607773 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.607791 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.607921 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.608394 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.608401 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.608687 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.608850 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.608859 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.610017 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.610021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.611129 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.611142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.611890 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.611894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.612013 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.612761 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.612778 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.613282 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.615735 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.615739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.615741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.616009 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.616367 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.616370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.616372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.616493 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.616967 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.616970 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.617090 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.617198 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.617457 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":117,"threads_started":1}
I20260807 06:12:21.618363 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.607414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.9.133.190:34569)
0807 06:12:21.607573 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.607578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.607863 (+ 285us) server_negotiation.cc:407] Connection header received
0807 06:12:21.608155 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.608159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.608220 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.608317 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.609001 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.609881 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.611318 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.611753 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.611868 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.613464 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.613472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.613478 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.613508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.615605 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.616123 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.616128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.616130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.616264 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.616578 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.616581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.616583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.616857 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.616973 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.618041 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.618188 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:12:21.619181 18314 tablet_copy_service.cc:140] P 8896cbe9e2a74deb83262b48f4d99d2d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 65e2a832f26d4cbc89f10e700713e982 ({username='oryx', principal='oryx/127.9.133.189@KRBTEST.COM'} at 127.0.0.1:49116)
I20260807 06:12:21.619448 18314 tablet_copy_service.cc:161] P 8896cbe9e2a74deb83262b48f4d99d2d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 65e2a832f26d4cbc89f10e700713e982 at {username='oryx', principal='oryx/127.9.133.189@KRBTEST.COM'} at 127.0.0.1:49116: session id = 65e2a832f26d4cbc89f10e700713e982-00000000000000000000000000000000
I20260807 06:12:21.621891 18314 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 06:12:21.624536 18926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 06:12:21.628777 18926 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: tablet copy: Starting download of 0 data blocks...
I20260807 06:12:21.629204 18926 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: tablet copy: Starting download of 1 WAL segments...
I20260807 06:12:21.632038 18926 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 06:12:21.635195 18314 tablet_copy_service.cc:342] P 8896cbe9e2a74deb83262b48f4d99d2d: Request end of tablet copy session 65e2a832f26d4cbc89f10e700713e982-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.133.189@KRBTEST.COM'} at 127.0.0.1:49116
I20260807 06:12:21.635428 18314 tablet_copy_service.cc:433] P 8896cbe9e2a74deb83262b48f4d99d2d: ending tablet copy session 65e2a832f26d4cbc89f10e700713e982-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 65e2a832f26d4cbc89f10e700713e982
I20260807 06:12:21.647693 18926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:21.650444 19091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:21.650714 18926 server_base.cc:1034] running on GCE node
W20260807 06:12:21.650509 19094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.651575 19092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:21.651865 18926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:21.652249 18926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:21.653412 18926 hybrid_clock.cc:648] HybridClock initialized: now 1786083141653400 us; error 41 us; skew 500 ppm
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:21.655645 18926 init.cc:377] Logged in from keytab as oryx/127.9.133.189@KRBTEST.COM (short username oryx)
I20260807 06:12:21.656289 18926 webserver.cc:460] Webserver started at http://127.9.133.189:34577/ using document root <none> and password file <none>
I20260807 06:12:21.656549 18926 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:21.656605 18926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:21.658336 18926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:21.659643 19103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.659919 18926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 06:12:21.660043 18926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
uuid: "65e2a832f26d4cbc89f10e700713e982"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "dee1bb7c44a1d458de7461584d53915c"
server_key_iv: "28c07b38bf4caa82287a1f784931e8b4"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.660213 18926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:21.675911 18926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:21.678730 18926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:21.685114 18926 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.189:38265
I20260807 06:12:21.685137 19177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.189:38265 every 8 connection(s)
I20260807 06:12:21.685961 18926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
I20260807 06:12:21.688331 19179 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:12:21.689601 19179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Bootstrap starting.
I20260807 06:12:21.702883 19179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:12:21.703354 19179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Bootstrap complete.
I20260807 06:12:21.704195 19179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } }
I20260807 06:12:21.704365 19179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 65e2a832f26d4cbc89f10e700713e982, State: Initialized, Role: LEARNER
I20260807 06:12:21.704488 19179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } }
I20260807 06:12:21.704919 19182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:21.705060 19179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:21.705157 19182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:12:21.709657 19198 catalog_manager.cc:1259] Loaded cluster ID: 30ab31c3798842e1b005d70a8070818e
I20260807 06:12:21.709754 19198 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: loading cluster ID for follower catalog manager: success
I20260807 06:12:21.712603 19198 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: acquiring CA information for follower catalog manager: success
I20260807 06:12:21.714730 19198 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 06:12:21.761540 19077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:21.761952 19077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:21.762029 19077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:21.766709 19077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:21.766808 19077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:21.766829 19077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:21.766852 19077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:21.766870 19077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:21.773545 19077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.188.keytab
--principal=oryx/127.9.133.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:37415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.133.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:21.775234 19077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:21.776640 19077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:21.787554 19211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.787559 19207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:21.787554 19204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:21.787986 19077 server_base.cc:1034] running on GCE node
I20260807 06:12:21.788810 19077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:21.790354 19077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:21.791592 19077 hybrid_clock.cc:648] HybridClock initialized: now 1786083141791562 us; error 37 us; skew 500 ppm
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:21.795411 19077 init.cc:377] Logged in from keytab as oryx/127.9.133.188@KRBTEST.COM (short username oryx)
I20260807 06:12:21.796685 19077 webserver.cc:460] Webserver started at http://127.9.133.188:35281/ using document root <none> and password file <none>
I20260807 06:12:21.797374 19077 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:21.797441 19077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:21.797652 19077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:21.800330 19077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/instance:
uuid: "122a8a0352a54dbea9688082e3735d25"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "45b5368578ef9f22d0e6bd14ae86ec60"
server_key_iv: "9ed6b0a0b4cdd539da1ecfd1f1493bfb"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.800994 19077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal/instance:
uuid: "122a8a0352a54dbea9688082e3735d25"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "45b5368578ef9f22d0e6bd14ae86ec60"
server_key_iv: "9ed6b0a0b4cdd539da1ecfd1f1493bfb"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.805876 19077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:21.809451 19222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:21.811304 19077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:21.811480 19077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
uuid: "122a8a0352a54dbea9688082e3735d25"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "45b5368578ef9f22d0e6bd14ae86ec60"
server_key_iv: "9ed6b0a0b4cdd539da1ecfd1f1493bfb"
server_key_version: "encryptionkey@0"
I20260807 06:12:21.811621 19077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:21.833992 19077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:21.838233 19077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:21.838582 19077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:21.850267 19077 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.188:37415
I20260807 06:12:21.850703 19296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.188:37415 every 8 connection(s)
I20260807 06:12:21.851701 19077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
I20260807 06:12:21.853008 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 19077
I20260807 06:12:21.855990 19297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:21.861223 19297 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.133.188@KRBTEST.COM
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.188@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:21.872077 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.855390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.9.133.188:37415)
0807 06:12:21.856024 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.856065 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.856113 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:12:21.857142 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.857165 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.857598 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.858039 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.859203 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.860377 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.861440 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.862083 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.862270 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.865674 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.865680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.865690 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.865766 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.867990 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.868510 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.868516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.868523 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.868713 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.869077 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.869081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.869083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.869721 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.870001 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.870318 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.870886 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":190,"threads_started":1}
I20260807 06:12:21.874747 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.863679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.9.133.190:34569)
0807 06:12:21.863851 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.863856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.864243 (+ 387us) server_negotiation.cc:407] Connection header received
0807 06:12:21.864447 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.864453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.864518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.864612 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.865645 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.866536 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.867916 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.868226 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.868341 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.871091 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.871095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.871099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.871135 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.872840 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.873359 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.873363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.873365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.873459 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.873747 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.873752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.873753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.873961 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.874078 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.874425 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.874561 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:12:21.874861 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.863493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49130)
0807 06:12:21.864103 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.864152 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.864309 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.864703 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.864710 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.865325 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.865525 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.865536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.866667 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.866672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.867769 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.867784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.868361 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.868364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.868521 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.869206 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.869219 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.870907 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.872959 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.872968 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.872974 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.873241 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.873556 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.873559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.873561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.873653 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.874089 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.874098 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.874220 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.874368 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.874629 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":211,"threads_started":1}
I20260807 06:12:21.876102 19297 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.188@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
I20260807 06:12:21.885557 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.876680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60804)
0807 06:12:21.876828 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.876843 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.876936 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.877560 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.877563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.877706 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.877876 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.877882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.878863 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.878866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.879483 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.879493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.880013 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.880015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.880089 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.880599 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.880611 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.881894 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.884033 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.884035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.884037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.884250 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.884540 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.884543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.884544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.884634 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.885047 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.885049 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.885134 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.885214 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.885403 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 06:12:21.885608 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.876916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.9.133.189:38265)
0807 06:12:21.877320 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.877324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.877338 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:21.877384 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.877386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.877432 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.877525 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.878000 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.878767 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.879598 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.879903 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.880023 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.882034 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.882037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.882040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.882064 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.883885 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.884343 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.884346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.884347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.884448 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.884727 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.884731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.884732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.884950 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.885040 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.885325 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.885465 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":179,"threads_started":1}
I20260807 06:12:21.886646 19297 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083141, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.188@KRBTEST.COM for oryx/127.9.133.188@KRBTEST.COM
I20260807 06:12:21.893713 18739 heartbeater.cc:503] Master 127.9.133.190:34569 was elected leader, sending a full tablet report...
I20260807 06:12:21.895545 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.887283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.0.0.1:36672)
0807 06:12:21.887479 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.887498 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.887586 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.887846 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.887849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.888023 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.888090 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.888095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.889413 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.889416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.889911 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.889920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.890382 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.890385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.890469 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.890901 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.890913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.892245 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.893800 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.893806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.893809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.894028 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.894398 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.894400 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.894402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.894542 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.894929 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.894932 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.895067 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.895201 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.895374 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 06:12:21.895648 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.887416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.9.133.188:37415)
0807 06:12:21.887549 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.887555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.887568 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:12:21.887692 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.887696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.887742 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.887838 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.888201 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.889193 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.890046 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.890274 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.890387 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.892375 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.892378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.892383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.892407 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.893693 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.894125 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.894132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.894134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.894248 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.894650 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.894653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.894655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.894827 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.894905 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.895275 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.895477 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":57}
I20260807 06:12:21.899430 19297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Bootstrap starting.
I20260807 06:12:21.902474 19297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:21.903513 19297 log.cc:824] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:21.906015 19297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: No bootstrap required, opened a new log
I20260807 06:12:21.910218 19297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } }
I20260807 06:12:21.910473 19297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:21.910552 19297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122a8a0352a54dbea9688082e3735d25, State: Initialized, Role: FOLLOWER
I20260807 06:12:21.911149 19297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } }
I20260807 06:12:21.911734 19306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } } }
I20260807 06:12:21.911904 19306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:21.912801 19297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 06:12:21.919883 19320 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:12:21.920048 19320 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:12:21.923875 19077 master_runner.cc:428] Detected that this master 122a8a0352a54dbea9688082e3735d25 is joining an existing cluster
I20260807 06:12:21.923954 19077 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.133.188:37415
I20260807 06:12:21.926000 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 18374
I20260807 06:12:21.943401 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:35435
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:34569
--webserver_port=35365
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:12:21.947633 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.933209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.9.133.190:34569)
0807 06:12:21.933521 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.933527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.936987 (+ 3460us) server_negotiation.cc:407] Connection header received
0807 06:12:21.937216 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.937223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.937299 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.937365 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.938247 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.939302 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.940516 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.940894 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.940981 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.942840 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.942845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.942848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.942881 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.945451 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.945967 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.945973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.945977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.946073 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.946390 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.946393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.946395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.946624 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.946699 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.947276 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.947427 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260807 06:12:21.948518 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.933737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49144)
0807 06:12:21.936866 (+ 3129us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.936896 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.937089 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.937554 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.937560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.937887 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.938061 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.938073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.939432 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.939435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.940365 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.940377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.941191 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.941195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.941443 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.942129 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.942145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.942671 (+ 526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.945589 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.945592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.945595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.945855 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.946183 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.946185 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.946188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.946277 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.946779 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.946782 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.946888 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.948139 (+ 1251us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.948301 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3038,"mutex_wait_us":83,"thread_start_us":119,"threads_started":1}
I20260807 06:12:21.954442 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.934301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.9.133.189:38265)
0807 06:12:21.934461 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.934466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.942491 (+ 8025us) server_negotiation.cc:407] Connection header received
0807 06:12:21.942660 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.942666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.942763 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.942850 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.943628 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.944571 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.946103 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.946562 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.946658 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.948392 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.948400 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.948406 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.948443 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.950645 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.951969 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.951976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.951981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.952104 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.952525 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.952532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.952537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.952889 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.953005 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.953946 (+ 941us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.954157 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:12:21.955300 19333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.934562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60808)
0807 06:12:21.942351 (+ 7789us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.942365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.942513 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.943031 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.943037 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.943298 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.943451 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.943460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.944758 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.944764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.945937 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.945953 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.946905 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.946911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.947058 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.947645 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.947664 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.948207 (+ 543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.950817 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.950823 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.950828 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.951201 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.952257 (+ 1056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.952264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.952268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.952385 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.953087 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.953092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.953223 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.954724 (+ 1501us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.954968 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7722,"thread_start_us":1362,"threads_started":1}
I20260807 06:12:21.964526 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.933331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.9.133.188:37415)
0807 06:12:21.933634 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.933638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.934249 (+ 611us) server_negotiation.cc:407] Connection header received
0807 06:12:21.934438 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.934442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.934491 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.934549 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.936744 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.937614 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.940943 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.941494 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.941611 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.960551 (+ 18940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.960555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.960559 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.960583 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.962659 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.963203 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.963207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.963209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.963286 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.963578 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.963581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.963583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.963800 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.963893 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.964169 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.964319 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":222}
I20260807 06:12:21.965492 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.933208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.0.0.1:36682)
0807 06:12:21.934155 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.934174 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.934332 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.934724 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.934727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.936242 (+ 1515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.936567 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.936583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.937778 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.937781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.940780 (+ 2999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.940796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.941756 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.941760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.941889 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.959706 (+ 17817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.959722 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.960394 (+ 672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.962785 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.962788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.962790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.963085 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.963390 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.963392 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.963395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.963486 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.963935 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.963937 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.964021 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.965133 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.965287 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":1,"thread_start_us":173,"threads_started":1}
Aug 07 06:12:21 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083140, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.190@KRBTEST.COM for oryx/127.9.133.188@KRBTEST.COM
I20260807 06:12:21.971668 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.953617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.0.0.1:36698)
0807 06:12:21.953778 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.953793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.953880 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.959793 (+ 5913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.959801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.960120 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:21.960334 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.960348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.962001 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.962006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.963089 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.963104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.963826 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.963830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.963964 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.964564 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:21.964582 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:21.966490 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:21.969674 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.969680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.969684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.969970 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.970371 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:21.970374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:21.970376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:21.970479 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:21.970943 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:21.970946 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:21.971146 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.971273 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.971461 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 06:12:21.971680 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.953669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.9.133.188:37415)
0807 06:12:21.959325 (+ 5656us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.959337 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.959353 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:12:21.959460 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.959467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.959560 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.960536 (+ 976us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:21.960707 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.961830 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.963264 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.963682 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.965943 (+ 2261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.966659 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:21.966663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:21.966666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:21.966694 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:21.969537 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.970099 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.970105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.970111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.970247 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:21.970594 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:21.970598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:21.970600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:21.970822 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:21.970925 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.971364 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.971496 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5562,"thread_start_us":125,"threads_started":1}
I20260807 06:12:21.973366 18288 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.133.188:37415: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "122a8a0352a54dbea9688082e3735d25"
member_type: NON_VOTER
last_known_addr {
host: "127.9.133.188"
port: 37415
}
attrs {
promote: true
}
}
dest_uuid: "8896cbe9e2a74deb83262b48f4d99d2d"
cas_config_opid_index: 5
I20260807 06:12:21.974294 18288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } }
I20260807 06:12:21.977083 19070 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 122a8a0352a54dbea9688082e3735d25 (127.9.133.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:21.979036 19070 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.133.188:37415
I20260807 06:12:21.979318 19070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:21.979436 19070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:21.980684 19260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:21.982530 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.976043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60820)
0807 06:12:21.976190 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:21.976208 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:21.976300 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:21.976945 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:21.976949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:21.976976 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:21.977111 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.977119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.978535 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.978540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.980506 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:21.980522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.981982 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.981986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:21.982100 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.982163 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 06:12:21.982242 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 06:12:21.982355 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 06:12:21.982571 19260 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Refusing update from remote peer 8896cbe9e2a74deb83262b48f4d99d2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 06:12:21.983255 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:21.976297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.9.133.189:38265)
0807 06:12:21.976537 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:21.976542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:21.976563 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:12:21.976635 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:21.976639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:21.976724 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:21.976861 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:21.977290 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.978218 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.980695 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:21.981485 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:21.981586 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:21.981806 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:21.982950 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0807 06:12:21.983099 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 06:12:21.983819 19071 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:21.984278 19145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 LEARNER]: Refusing update from remote peer 8896cbe9e2a74deb83262b48f4d99d2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 06:12:21.984723 19070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 06:12:21.989048 19260 raft_consensus.cc:858] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } }
W20260807 06:12:21.989162 19260 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } }
I20260807 06:12:21.991290 19338 mvcc.cc:205] Tried to move back new op lower bound from 7315796542566965248 to 7315796542519681024. Current Snapshot: MvccSnapshot[applied={T|T < 7315796542566965248}]
I20260807 06:12:21.991746 19077 master.cc:537] Master@127.9.133.188:37415 shutting down...
I20260807 06:12:21.992645 19145 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 122a8a0352a54dbea9688082e3735d25 (127.9.133.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:21.995189 19182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8896cbe9e2a74deb83262b48f4d99d2d. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:21.995319 19182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:12:21.995404 19182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:21.995463 19182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:12:21.996284 19260 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:21.998106 19070 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: attempting to promote NON_VOTER 65e2a832f26d4cbc89f10e700713e982 to VOTER
I20260807 06:12:21.998695 19070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } }
I20260807 06:12:21.999662 19343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:21.999797 19343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 06:12:21.999892 19343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:21.999960 19343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 06:12:22.000233 19306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8896cbe9e2a74deb83262b48f4d99d2d. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: NON_VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: true } } }
I20260807 06:12:22.000464 19306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260807 06:12:22.002418 18260 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d -> Peer 122a8a0352a54dbea9688082e3735d25 (127.9.133.188:37415): Couldn't send request to peer 122a8a0352a54dbea9688082e3735d25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 06:12:22.004472 19077 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 06:12:22.004854 19077 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 06:12:22.004990 19077 tablet_replica.cc:330] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: stopping tablet replica
I20260807 06:12:22.005786 19145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 LEARNER]: Refusing update from remote peer 8896cbe9e2a74deb83262b48f4d99d2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 06:12:22.008242 19070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 06:12:22.011832 19071 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.014072 19071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.014235 19071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:22.015422 19071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.015625 19071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:22.015935 19145 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.017835 19346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.018002 19346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:22.020009 19077 master.cc:552] Master@127.9.133.188:37415 shutdown complete.
I20260807 06:12:22.020100 19077 master_runner.cc:305] Clearing existing system tablet
I20260807 06:12:22.020401 19077 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:22.020489 19077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:22.021865 19077 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 06:12:22.022897 19356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.023229 19077 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:22.023401 19077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:22.023442 19077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
uuid: "122a8a0352a54dbea9688082e3735d25"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "45b5368578ef9f22d0e6bd14ae86ec60"
server_key_iv: "9ed6b0a0b4cdd539da1ecfd1f1493bfb"
server_key_version: "encryptionkey@0"
I20260807 06:12:22.025738 19077 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:22.028398 19077 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 06:12:22.028519 19077 log.cc:1196] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 06:12:22.028888 19077 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Deleting consensus metadata
I20260807 06:12:22.029119 19077 master_runner.cc:315] Copying system tablet from 127.9.133.190:34569
I20260807 06:12:22.030575 19077 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: tablet copy: Beginning tablet copy session from remote peer at address 127.9.133.190:34569
I20260807 06:12:22.039760 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.031354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49158)
0807 06:12:22.031790 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.031804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.031975 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.032296 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.032300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.032569 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.032695 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.032701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.033567 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.033572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.034184 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.034192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.034705 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.034709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.034817 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.035532 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.035555 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.035973 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.037979 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.037985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.037987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.038218 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.038525 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.038529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.038531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.038627 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.039112 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.039115 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.039236 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.039357 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.039537 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":107,"threads_started":1}
I20260807 06:12:22.039760 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.031482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.9.133.190:34569)
0807 06:12:22.031650 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.031655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.031911 (+ 256us) server_negotiation.cc:407] Connection header received
0807 06:12:22.032108 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.032112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.032183 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.032290 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.032838 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.033444 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.034323 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.034572 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.034697 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.036126 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.036130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.036133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.036170 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.037858 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.038327 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.038332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.038336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.038430 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.038724 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.038728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.038730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.038954 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.039117 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.039426 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.039559 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:12:22.040782 18314 tablet_copy_service.cc:140] P 8896cbe9e2a74deb83262b48f4d99d2d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 122a8a0352a54dbea9688082e3735d25 ({username='oryx', principal='oryx/127.9.133.188@KRBTEST.COM'} at 127.0.0.1:49158)
I20260807 06:12:22.040925 18314 tablet_copy_service.cc:161] P 8896cbe9e2a74deb83262b48f4d99d2d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 122a8a0352a54dbea9688082e3735d25 at {username='oryx', principal='oryx/127.9.133.188@KRBTEST.COM'} at 127.0.0.1:49158: session id = 122a8a0352a54dbea9688082e3735d25-00000000000000000000000000000000
I20260807 06:12:22.041992 18314 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 06:12:22.043506 19077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 06:12:22.045815 19077 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: tablet copy: Starting download of 0 data blocks...
I20260807 06:12:22.046113 19077 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: tablet copy: Starting download of 1 WAL segments...
I20260807 06:12:22.047799 19077 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 06:12:22.049502 18314 tablet_copy_service.cc:342] P 8896cbe9e2a74deb83262b48f4d99d2d: Request end of tablet copy session 122a8a0352a54dbea9688082e3735d25-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.133.188@KRBTEST.COM'} at 127.0.0.1:49158
I20260807 06:12:22.049618 18314 tablet_copy_service.cc:433] P 8896cbe9e2a74deb83262b48f4d99d2d: ending tablet copy session 122a8a0352a54dbea9688082e3735d25-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 122a8a0352a54dbea9688082e3735d25
I20260807 06:12:22.058174 19077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:22.060215 19365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.060281 19372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.060786 19368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:22.064400 19077 server_base.cc:1034] running on GCE node
I20260807 06:12:22.064635 19077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:22.065155 19077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:22.066344 19077 hybrid_clock.cc:648] HybridClock initialized: now 1786083142066328 us; error 49 us; skew 500 ppm
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:22.068302 19077 init.cc:377] Logged in from keytab as oryx/127.9.133.188@KRBTEST.COM (short username oryx)
I20260807 06:12:22.068930 19077 webserver.cc:460] Webserver started at http://127.9.133.188:36499/ using document root <none> and password file <none>
I20260807 06:12:22.069157 19077 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:22.069216 19077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:22.070590 19077 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 06:12:22.071825 19381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.072113 19077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:22.072207 19077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
uuid: "122a8a0352a54dbea9688082e3735d25"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "45b5368578ef9f22d0e6bd14ae86ec60"
server_key_iv: "9ed6b0a0b4cdd539da1ecfd1f1493bfb"
server_key_version: "encryptionkey@0"
I20260807 06:12:22.072333 19077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:22.082787 19077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:22.085702 19077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:22.092062 19077 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.188:37415
I20260807 06:12:22.092257 19454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.188:37415 every 8 connection(s)
I20260807 06:12:22.092819 19077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
I20260807 06:12:22.094708 19455 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:12:22.095862 19455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Bootstrap starting.
I20260807 06:12:22.103844 19455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:12:22.104234 19455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: Bootstrap complete.
I20260807 06:12:22.104925 19455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } }
I20260807 06:12:22.105175 19455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 122a8a0352a54dbea9688082e3735d25, State: Initialized, Role: LEARNER
I20260807 06:12:22.105324 19455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } }
I20260807 06:12:22.105760 19458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.106019 19455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:22.106016 19458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:12:22.110572 19472 catalog_manager.cc:1259] Loaded cluster ID: 30ab31c3798842e1b005d70a8070818e
I20260807 06:12:22.110730 19472 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: loading cluster ID for follower catalog manager: success
I20260807 06:12:22.112892 19472 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: acquiring CA information for follower catalog manager: success
I20260807 06:12:22.114252 19472 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 06:12:22.141896 19334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:22.142452 19334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:22.142531 19334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:22.149430 19334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:22.149540 19334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:22.149641 19334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:22.156188 19334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:35435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=35365
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:22.157608 19334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:22.159032 19334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:22.171968 19477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.171988 19479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.171941 19481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:22.172446 19334 server_base.cc:1034] running on GCE node
I20260807 06:12:22.173022 19334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:22.173935 19334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:22.175114 19334 hybrid_clock.cc:648] HybridClock initialized: now 1786083142175094 us; error 38 us; skew 500 ppm
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:22.178647 19334 init.cc:377] Logged in from keytab as oryx/127.9.133.129@KRBTEST.COM (short username oryx)
I20260807 06:12:22.179790 19334 webserver.cc:460] Webserver started at http://127.9.133.129:35365/ using document root <none> and password file <none>
I20260807 06:12:22.180449 19334 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:22.180532 19334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:22.186057 19334 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:22.189023 19489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.190513 19334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:22.190670 19334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "0c8e26a4d040484fbb0967275acfa7ab"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "d4380fd78905b4ddf91e91bd11a60cd8"
server_key_iv: "5f0b0567af25e186f3aa6b854e421146"
server_key_version: "encryptionkey@0"
I20260807 06:12:22.191414 19334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:22.209457 18920 heartbeater.cc:503] Master 127.9.133.190:34569 was elected leader, sending a full tablet report...
I20260807 06:12:22.210170 19334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:22.214833 19334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:22.215127 19334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:22.215993 19334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:22.217314 19334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:22.217378 19334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.217463 19334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:22.217499 19334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.229945 19334 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:35435
I20260807 06:12:22.229970 19652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:35435 every 8 connection(s)
I20260807 06:12:22.231474 19334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:22.240417 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 19334
I20260807 06:12:22.240706 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 18563
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.188@KRBTEST.COM
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
I20260807 06:12:22.267150 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.234359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:57739 (local address 127.9.133.188:37415)
0807 06:12:22.234720 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.234727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.235230 (+ 503us) server_negotiation.cc:407] Connection header received
0807 06:12:22.236661 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.236665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.236739 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.236809 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.239081 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.240203 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.245555 (+ 5352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.246120 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.246237 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.261104 (+ 14867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.261109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.261112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.261168 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.263899 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.264733 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.264739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.264743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.264840 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265402 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.265408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.265413 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.265701 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.265795 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.266749 (+ 954us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.266912 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
I20260807 06:12:22.267146 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.236072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:49607 (local address 127.9.133.190:34569)
0807 06:12:22.236235 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.236241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.236260 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:12:22.236655 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.236659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.236726 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.236785 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.239081 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.240001 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.243821 (+ 3820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.244337 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.244443 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.261628 (+ 17185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.261634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.261639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.261683 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.264449 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265055 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.265059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.265061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.265169 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265503 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.265507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.265508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.265734 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.265844 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.266749 (+ 905us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.266911 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 06:12:22.267519 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.234006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:58655 (local address 127.9.133.189:38265)
0807 06:12:22.234210 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.234216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.235724 (+ 1508us) server_negotiation.cc:407] Connection header received
0807 06:12:22.236757 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.236762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.236830 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.236901 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.239081 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.239979 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.256505 (+ 16526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.256960 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.257063 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.262527 (+ 5464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.262536 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.262543 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.262587 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.264952 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265609 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.265616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.265621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.265791 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.266155 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.266162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.266167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.266450 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.266548 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.266976 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.267236 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 06:12:22.269275 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:46105
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:34569
--webserver_port=38897
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:12:22.269383 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.234411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.9.133.129:58655)
0807 06:12:22.235625 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.235638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.236389 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.237164 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.237174 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.237906 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.238859 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.238881 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.255139 (+ 16258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.255144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.256272 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.256286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.257291 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.257295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.257421 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.259398 (+ 1977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.259423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.262296 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.265109 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.265122 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.265431 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.265930 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265937 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.265942 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.266046 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.266618 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.266624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.266742 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.267786 (+ 1044us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.268793 (+ 1007us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":918,"spinlock_wait_cycles":50176,"thread_start_us":209,"threads_started":1}
I20260807 06:12:22.269377 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.235682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:49607)
0807 06:12:22.236146 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.236161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.236401 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.237164 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.237174 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.237890 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.238867 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.238883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.240296 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.240300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.243643 (+ 3343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.243659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.244742 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.244750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.245067 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.247118 (+ 2051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.247156 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.261384 (+ 14228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.264606 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.264611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.264613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.264933 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.265283 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.265286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.265287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.265383 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.265920 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.265922 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.266405 (+ 483us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.267464 (+ 1059us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.267969 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":1907,"spinlock_wait_cycles":44160,"thread_start_us":91,"threads_started":1}
I20260807 06:12:22.269374 19659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.234411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.9.133.129:57739)
0807 06:12:22.235042 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.235102 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.236389 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.237164 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.237174 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.237906 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.238859 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.238881 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.240351 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.240354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.242429 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.242443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.246466 (+ 4023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.246472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.246675 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.248462 (+ 1787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.248487 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.260809 (+ 12322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.264104 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.264114 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.264144 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.264606 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.264973 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.264978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.264981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.265292 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.265874 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.265886 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.266406 (+ 520us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.267954 (+ 1548us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.268131 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":295,"spinlock_wait_cycles":33664,"thread_start_us":157,"threads_started":1}
I20260807 06:12:22.271374 19653 heartbeater.cc:347] Connected to a master server at 127.9.133.188:37415
I20260807 06:12:22.271448 19654 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34569
I20260807 06:12:22.271751 19654 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.271750 19653 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.271422 19655 heartbeater.cc:347] Connected to a master server at 127.9.133.189:38265
I20260807 06:12:22.272431 19655 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.272677 19653 heartbeater.cc:511] Master 127.9.133.188:37415 requested a full tablet report, sending...
I20260807 06:12:22.272692 19655 heartbeater.cc:511] Master 127.9.133.189:38265 requested a full tablet report, sending...
I20260807 06:12:22.274044 19654 heartbeater.cc:511] Master 127.9.133.190:34569 requested a full tablet report, sending...
I20260807 06:12:22.275098 18288 ts_manager.cc:194] Re-registered known tserver with Master: 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435)
I20260807 06:12:22.275090 19132 ts_manager.cc:194] Registered new tserver with Master: 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435)
I20260807 06:12:22.275125 19401 ts_manager.cc:194] Registered new tserver with Master: 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435)
I20260807 06:12:22.276024 18288 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:49607
W20260807 06:12:22.413968 19662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:22.414305 19662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:22.414356 19662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:22.419602 19662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:22.419729 19662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:22.419826 19662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:22.421047 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.416552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.0.0.1:36702)
0807 06:12:22.416729 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.416750 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.416918 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.417299 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.417302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.417326 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:22.417453 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.417468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.418615 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.418618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.419561 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.419573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.420633 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.420636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.420714 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.420739 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.420788 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.420898 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 06:12:22.421047 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.416675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.9.133.188:37415)
0807 06:12:22.416862 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.416868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.416890 (+ 22us) server_negotiation.cc:407] Connection header received
0807 06:12:22.417049 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.417053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.417135 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.417255 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:22.417581 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.418481 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.419727 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.420478 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.420590 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.420779 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.420819 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.420905 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:12:22.421941 19414 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 06:12:22.422300 19458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8896cbe9e2a74deb83262b48f4d99d2d. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.422505 19458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: LEARNER
I20260807 06:12:22.426445 19662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:46105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=38897
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:22.427943 19662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:22.429282 19662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:22.441049 19669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.441043 19673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.441039 19671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:22.441371 19662 server_base.cc:1034] running on GCE node
I20260807 06:12:22.441973 19662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:22.442821 19662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:22.444031 19662 hybrid_clock.cc:648] HybridClock initialized: now 1786083142443985 us; error 56 us; skew 500 ppm
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:22.447521 19662 init.cc:377] Logged in from keytab as oryx/127.9.133.130@KRBTEST.COM (short username oryx)
I20260807 06:12:22.448622 19662 webserver.cc:460] Webserver started at http://127.9.133.130:38897/ using document root <none> and password file <none>
I20260807 06:12:22.449334 19662 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:22.449473 19662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:22.454294 19662 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:22.456955 19683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.458401 19662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 06:12:22.458566 19662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "fb6649c6dea548c98d4bedc1b73d852e"
format_stamp: "Formatted at 2026-08-07 06:12:20 on dist-test-slave-h3ft"
server_key: "68effc671786a64bba8991a6b7dcb16f"
server_key_iv: "8e8ce919935a650c9fded8bce4c0acbe"
server_key_version: "encryptionkey@0"
I20260807 06:12:22.459313 19662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:22.476249 19662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:22.480369 19662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:22.480686 19662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:22.481698 19662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:22.482827 19662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:22.482879 19662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.482926 19662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:22.482944 19662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.493745 19662 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:46105
I20260807 06:12:22.493767 19852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:46105 every 8 connection(s)
I20260807 06:12:22.495208 19662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:22.501787 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 19662
I20260807 06:12:22.502058 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 18744
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.188@KRBTEST.COM
I20260807 06:12:22.516355 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.498069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:50173 (local address 127.9.133.188:37415)
0807 06:12:22.498214 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.498225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.499390 (+ 1165us) server_negotiation.cc:407] Connection header received
0807 06:12:22.501067 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.501071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.501165 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.501259 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.503255 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.503850 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.505847 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.506445 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.506563 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.510947 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.510954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.510961 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.511031 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.512795 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.513597 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.513600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.513602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.513697 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.514239 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.514243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.514244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.514494 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.514586 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.515484 (+ 898us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.515663 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
I20260807 06:12:22.517437 19857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.498468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.9.133.130:50173)
0807 06:12:22.499165 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.499239 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.500599 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.501395 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.501406 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.502015 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.502937 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.502957 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.504031 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.504037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.505638 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.505654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.506992 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.507021 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.507358 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.508429 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.508460 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.510684 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.512945 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.512956 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.512984 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.513471 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.513794 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.513800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.513805 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.514075 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.514613 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.514624 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.515132 (+ 508us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.515388 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.515886 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":3,"spinlock_wait_cycles":48384,"thread_start_us":171,"threads_started":1}
I20260807 06:12:22.520005 19853 heartbeater.cc:347] Connected to a master server at 127.9.133.188:37415
I20260807 06:12:22.520363 19853 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.520745 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:39479
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:34569
--webserver_port=39749
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:12:22.521167 19853 heartbeater.cc:511] Master 127.9.133.188:37415 requested a full tablet report, sending...
I20260807 06:12:22.522760 19401 ts_manager.cc:194] Registered new tserver with Master: fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:22.525864 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.500006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34587 (local address 127.9.133.189:38265)
0807 06:12:22.500186 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.500192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.500286 (+ 94us) server_negotiation.cc:407] Connection header received
0807 06:12:22.501338 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.501342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.501405 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.501492 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.505859 (+ 4367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.506601 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.508167 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.508588 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.508687 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.517905 (+ 9218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.517911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.517914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.517954 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.520033 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.524094 (+ 4061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.524103 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.524107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.524244 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.524628 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.524635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.524639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.524947 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.525055 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.525484 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.525666 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 06:12:22.525882 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.499857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.9.133.130:34587)
0807 06:12:22.500174 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.500195 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.500626 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.501552 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.501555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.502028 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.505595 (+ 3567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.505608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.506778 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.506783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.507927 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.507942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.508941 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.508947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.509099 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.515169 (+ 6070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.515193 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.517685 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.523552 (+ 5867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.523558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.523560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.523926 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.524376 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.524382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.524385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.524492 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.525128 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.525130 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.525245 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.525370 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.525578 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":3165,"spinlock_wait_cycles":5760,"thread_start_us":133,"threads_started":1}
I20260807 06:12:22.526844 19855 heartbeater.cc:347] Connected to a master server at 127.9.133.189:38265
I20260807 06:12:22.527107 19855 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.527521 19855 heartbeater.cc:511] Master 127.9.133.189:38265 requested a full tablet report, sending...
I20260807 06:12:22.528384 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.498468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:38083)
0807 06:12:22.499165 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.499239 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.500599 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.516811 (+ 16212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.516818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.517032 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.517193 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.517199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.519348 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.519354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.520357 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.520369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.521287 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.521293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.521421 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.521965 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.521987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.524009 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.526262 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.526268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.526273 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.526581 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.526968 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.526972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.526974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.527169 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.527800 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.527805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.527911 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.528007 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.528178 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":33,"spinlock_wait_cycles":48768,"thread_start_us":186,"threads_started":1}
I20260807 06:12:22.528704 19132 ts_manager.cc:194] Registered new tserver with Master: fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:22.529033 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.497800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:38083 (local address 127.9.133.190:34569)
0807 06:12:22.497990 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.497996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.499404 (+ 1408us) server_negotiation.cc:407] Connection header received
0807 06:12:22.500951 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.500957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.501036 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.501121 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.517367 (+ 16246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.518291 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.520556 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.520939 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.521049 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.524205 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.524210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.524213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.524249 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.526127 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.526720 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.526724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.526726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.526845 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.527297 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.527300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.527302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.527629 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.527745 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.528655 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.528812 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:12:22.529738 19854 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34569
I20260807 06:12:22.529866 19854 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.530194 19854 heartbeater.cc:511] Master 127.9.133.190:34569 requested a full tablet report, sending...
I20260807 06:12:22.531157 18288 ts_manager.cc:194] Re-registered known tserver with Master: fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:22.532191 18288 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:38083
W20260807 06:12:22.660226 19863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:22.660557 19863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:22.660604 19863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:22.664920 19863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:22.664986 19863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:22.665047 19863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:22.670572 19863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:39479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=39749
--tserver_master_addrs=127.9.133.190:34569,127.9.133.189:38265,127.9.133.188:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:22.671836 19863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:22.673254 19863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:22.684998 19876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:22.685132 19871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:22.685259 19863 server_base.cc:1034] running on GCE node
W20260807 06:12:22.685045 19870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:22.687664 19863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:22.688496 19863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:22.689695 19863 hybrid_clock.cc:648] HybridClock initialized: now 1786083142689662 us; error 49 us; skew 500 ppm
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:22.693207 19863 init.cc:377] Logged in from keytab as oryx/127.9.133.131@KRBTEST.COM (short username oryx)
I20260807 06:12:22.694454 19863 webserver.cc:460] Webserver started at http://127.9.133.131:39749/ using document root <none> and password file <none>
I20260807 06:12:22.695328 19863 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:22.695467 19863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:22.700583 19863 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:22.703313 19887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.704979 19863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:22.705166 19863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "a95e253980cf47a1ab96894a38a3d371"
format_stamp: "Formatted at 2026-08-07 06:12:21 on dist-test-slave-h3ft"
server_key: "03535c9cbdf38dd86caca5c489716a28"
server_key_iv: "977573786064f5e6ab3ba3b1418b7797"
server_key_version: "encryptionkey@0"
I20260807 06:12:22.705922 19863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:22.737466 19863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:22.741662 19863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:22.741891 19863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:22.742662 19863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:22.743989 19863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:22.744055 19863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.744109 19863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:22.744139 19863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:22.756520 19863 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:39479
I20260807 06:12:22.756543 20059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:39479 every 8 connection(s)
I20260807 06:12:22.757905 19863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:22.764433 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 19863
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.188@KRBTEST.COM
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.189@KRBTEST.COM
Aug 07 06:12:22 dist-test-slave-h3ft krb5kdc[18217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083142, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:22.783844 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.760451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:56041 (local address 127.9.133.188:37415)
0807 06:12:22.760644 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.760650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.762129 (+ 1479us) server_negotiation.cc:407] Connection header received
0807 06:12:22.764087 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.764094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.764201 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.764307 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.768086 (+ 3779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.769052 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.772524 (+ 3472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.772991 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.773087 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.777754 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.777761 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.777768 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.777806 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.780181 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.781400 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.781404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.781407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.781522 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.782080 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.782084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.782087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.782402 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.782503 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.783466 (+ 963us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.783624 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:12:22.785211 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.761095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.9.133.131:56041)
0807 06:12:22.761829 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.761912 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.763324 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.764334 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.764341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.767685 (+ 3344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.767859 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.767871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.769323 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.769331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.772359 (+ 3028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.772372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.773352 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.773356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.773625 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.774587 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.774612 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.777523 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.780714 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.780727 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.780766 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.781256 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.781679 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.781683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.781685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.781946 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.782585 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.782598 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.783110 (+ 512us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.783392 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.783843 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":31,"spinlock_wait_cycles":84224,"thread_start_us":193,"threads_started":1}
I20260807 06:12:22.788298 20062 heartbeater.cc:347] Connected to a master server at 127.9.133.188:37415
I20260807 06:12:22.788661 20062 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.789067 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.761095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:52591)
0807 06:12:22.761829 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.761906 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.763330 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.764134 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.764168 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.765044 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.766089 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.766115 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.772459 (+ 6344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.772465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.773527 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.773536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.779155 (+ 5619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.779163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.779370 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.780088 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.780109 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.782230 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.785505 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.785510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.785512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.785867 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.787527 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.787530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.787532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.787630 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.788465 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.788469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.788578 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.788685 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.788864 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"spinlock_wait_cycles":4352,"thread_start_us":182,"threads_started":1}
I20260807 06:12:22.789480 20062 heartbeater.cc:511] Master 127.9.133.188:37415 requested a full tablet report, sending...
I20260807 06:12:22.789759 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.760451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:52591 (local address 127.9.133.190:34569)
0807 06:12:22.760644 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.760650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.762066 (+ 1416us) server_negotiation.cc:407] Connection header received
0807 06:12:22.763619 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.763623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.763701 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.763834 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.766277 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.767118 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.775298 (+ 8180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.775945 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.776180 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.782417 (+ 6237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.782423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.782430 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.782473 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.785358 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.787114 (+ 1756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.787122 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.787125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.787252 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.787762 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.787767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.787769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.788029 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.788130 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.788893 (+ 763us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.789067 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 06:12:22.789929 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.762641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:33567 (local address 127.9.133.189:38265)
0807 06:12:22.762808 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.762814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.763027 (+ 213us) server_negotiation.cc:407] Connection header received
0807 06:12:22.763616 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.763620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.763698 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.763880 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.771073 (+ 7193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.771908 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.773808 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.774223 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.774332 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.782701 (+ 8369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.782706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.782709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.782748 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.786661 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.787278 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.787282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.787284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.787390 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.787950 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.787953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.787955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.788249 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.788352 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.789606 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.789757 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:12:22.790477 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.762511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.9.133.131:33567)
0807 06:12:22.762908 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.762922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.763345 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.767123 (+ 3778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.767126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.770619 (+ 3493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:22.770815 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.770834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.772205 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.772214 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.773292 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.773308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.774594 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.774598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.774732 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.776433 (+ 1701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:22.776458 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:22.782494 (+ 6036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:22.786795 (+ 4301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.786797 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.786799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.787143 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.787736 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:22.787739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:22.787741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:22.787824 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:22.789266 (+ 1442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:22.789269 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:22.789361 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.790090 (+ 729us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.790268 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":129,"threads_started":1}
I20260807 06:12:22.790797 20064 heartbeater.cc:347] Connected to a master server at 127.9.133.190:34569
I20260807 06:12:22.790920 20064 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.791169 20064 heartbeater.cc:511] Master 127.9.133.190:34569 requested a full tablet report, sending...
I20260807 06:12:22.791249 20065 heartbeater.cc:347] Connected to a master server at 127.9.133.189:38265
I20260807 06:12:22.791352 20065 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:22.791440 19401 ts_manager.cc:194] Registered new tserver with Master: a95e253980cf47a1ab96894a38a3d371 (127.9.133.131:39479)
I20260807 06:12:22.791575 20065 heartbeater.cc:511] Master 127.9.133.189:38265 requested a full tablet report, sending...
I20260807 06:12:22.792110 18288 ts_manager.cc:194] Re-registered known tserver with Master: a95e253980cf47a1ab96894a38a3d371 (127.9.133.131:39479)
I20260807 06:12:22.792541 19132 ts_manager.cc:194] Registered new tserver with Master: a95e253980cf47a1ab96894a38a3d371 (127.9.133.131:39479)
I20260807 06:12:22.793011 18288 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:52591
I20260807 06:12:22.797245 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.771595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.9.133.188:37415)
0807 06:12:22.774631 (+ 3036us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.774636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.775044 (+ 408us) server_negotiation.cc:407] Connection header received
0807 06:12:22.775286 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.775294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.775369 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.775466 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.781099 (+ 5633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.782160 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.784763 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.785368 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.786023 (+ 655us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.792010 (+ 5987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.792015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.792018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.792048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.794339 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.794942 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.794949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.794954 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.795104 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.795559 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.795566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.795571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.795860 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.796031 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.796831 (+ 800us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.796991 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2943,"thread_start_us":104,"threads_started":1}
I20260807 06:12:22.801503 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.770841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.9.133.190:34569)
0807 06:12:22.776274 (+ 5433us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.776291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.776308 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:12:22.776351 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.776354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.776418 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.776533 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.787516 (+ 10983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.788353 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.794116 (+ 5763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.794511 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.794629 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.797152 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.797155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.797158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.797182 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.799451 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.799937 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.799943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.799948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.800066 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.800388 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.800391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.800392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.800701 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.800825 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.801125 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.801289 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5311,"thread_start_us":150,"threads_started":1}
I20260807 06:12:22.802261 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.771865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.9.133.189:38265)
0807 06:12:22.774430 (+ 2565us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.774437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.776915 (+ 2478us) server_negotiation.cc:407] Connection header received
0807 06:12:22.785604 (+ 8689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.785610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.785698 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.785795 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:22.789960 (+ 4165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.791586 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.795075 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.795536 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.795668 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.798155 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:22.798162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:22.798167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:22.798205 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:22.800006 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.800503 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.800506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.800508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.800599 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:22.800910 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:22.800913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:22.800915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:22.801198 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:22.801330 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.801995 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.802115 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2438,"thread_start_us":139,"threads_started":1}
I20260807 06:12:22.805091 18288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:22.810043 18288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:22.813943 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: NON_VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: true } } }
I20260807 06:12:22.814157 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:22.816888 19353 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: attempting to promote NON_VOTER 122a8a0352a54dbea9688082e3735d25 to VOTER
I20260807 06:12:22.817368 19353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:22.818823 19414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 LEARNER]: Refusing update from remote peer 8896cbe9e2a74deb83262b48f4d99d2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 06:12:22.818856 19145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Refusing update from remote peer 8896cbe9e2a74deb83262b48f4d99d2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 06:12:22.819334 19353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 06:12:22.819520 20088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 06:12:22.821580 20086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 122a8a0352a54dbea9688082e3735d25 (127.9.133.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.822115 19414 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 122a8a0352a54dbea9688082e3735d25 (127.9.133.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.822314 19145 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 122a8a0352a54dbea9688082e3735d25 (127.9.133.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.823623 20086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.823729 20086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:22.823658 20088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.823789 20088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:22.823874 20086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.823966 20086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:22.824575 20098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.824734 20098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:22.827483 19458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:22.827641 19458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:22.839130 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.832757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:39479 (local address 127.0.0.1:57072)
0807 06:12:22.833334 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.833357 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.833465 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.834492 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.834495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.834533 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:22.834750 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.834763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.836252 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.836255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.837497 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.837509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.838631 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.838635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.838726 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.838759 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.838834 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.838912 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":110,"spinlock_wait_cycles":7680}
I20260807 06:12:22.839288 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.833049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.9.133.131:39479)
0807 06:12:22.833834 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.833849 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.833899 (+ 50us) server_negotiation.cc:407] Connection header received
0807 06:12:22.834008 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.834016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.834309 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.834530 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:22.834924 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.836150 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.837640 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.838525 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.838631 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.838887 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.838951 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.839102 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":137,"threads_started":1}
I20260807 06:12:22.840423 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.832978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:35435 (local address 127.0.0.1:45546)
0807 06:12:22.833706 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.833721 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.833835 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.834494 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.834497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.834535 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:22.834747 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.834760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.836159 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.836164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.838362 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.838376 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.839968 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.839972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.840053 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.840072 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.840172 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.840257 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":128,"spinlock_wait_cycles":6016,"thread_start_us":157,"threads_started":1}
I20260807 06:12:22.840669 20103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.833195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45546 (local address 127.9.133.129:35435)
0807 06:12:22.833605 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.833615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.833961 (+ 346us) server_negotiation.cc:407] Connection header received
0807 06:12:22.834070 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.834076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.834315 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.834531 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:22.834895 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.836041 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.838525 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.839617 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.839733 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.839896 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.840222 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.840347 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":79,"threads_started":1}
I20260807 06:12:22.841243 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.833750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.9.133.130:46105)
0807 06:12:22.834237 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:22.834249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:22.835203 (+ 954us) server_negotiation.cc:407] Connection header received
0807 06:12:22.835327 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:22.835332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:22.835534 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:22.835693 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:22.835956 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.836837 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.838026 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.838851 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.838975 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.839147 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:22.840597 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0807 06:12:22.840701 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":147,"threads_started":1}
I20260807 06:12:22.840478 20105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:22.833274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:46105 (local address 127.0.0.1:57950)
0807 06:12:22.835071 (+ 1797us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:22.835087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:22.835193 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:22.835691 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:22.835695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:22.835715 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:22.835826 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.835839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.836960 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.836966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.837872 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:22.837894 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:22.839216 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:22.839219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:22.839321 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:22.839355 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 06:12:22.840249 (+ 894us) client_negotiation.cc:240] Negotiation successful
0807 06:12:22.840337 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1674,"mutex_wait_us":1,"thread_start_us":142,"threads_started":1}
I20260807 06:12:22.843281 19958 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba3c917c833415086998b16bfad039a (DEFAULT_TABLE table=test-table [id=98a429b511754caf95a2567da6625de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:22.844794 19958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba3c917c833415086998b16bfad039a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:22.844604 19754 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba3c917c833415086998b16bfad039a (DEFAULT_TABLE table=test-table [id=98a429b511754caf95a2567da6625de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:22.845783 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba3c917c833415086998b16bfad039a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:22.854558 19557 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba3c917c833415086998b16bfad039a (DEFAULT_TABLE table=test-table [id=98a429b511754caf95a2567da6625de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:22.855923 20108 tablet_bootstrap.cc:492] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Bootstrap starting.
I20260807 06:12:22.856065 19557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba3c917c833415086998b16bfad039a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:22.859352 20108 tablet_bootstrap.cc:654] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:22.859398 20110 tablet_bootstrap.cc:492] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Bootstrap starting.
I20260807 06:12:22.860457 20108 log.cc:824] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:22.862046 20110 tablet_bootstrap.cc:654] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:22.862973 20110 log.cc:824] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:22.863248 20108 tablet_bootstrap.cc:492] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: No bootstrap required, opened a new log
I20260807 06:12:22.863497 20108 ts_tablet_manager.cc:1397] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 06:12:22.865612 20110 tablet_bootstrap.cc:492] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: No bootstrap required, opened a new log
I20260807 06:12:22.865844 20110 ts_tablet_manager.cc:1397] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260807 06:12:22.866443 20113 tablet_bootstrap.cc:492] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Bootstrap starting.
I20260807 06:12:22.869279 20113 tablet_bootstrap.cc:654] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:22.869470 20108 raft_consensus.cc:348] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:22.869788 20108 raft_consensus.cc:374] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:22.869848 20108 raft_consensus.cc:729] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6649c6dea548c98d4bedc1b73d852e, State: Initialized, Role: FOLLOWER
I20260807 06:12:22.870389 20108 consensus_queue.cc:260] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:22.870642 20113 log.cc:824] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:22.871691 20108 ts_tablet_manager.cc:1428] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:12:22.871469 20110 raft_consensus.cc:348] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:22.871841 20110 raft_consensus.cc:374] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:22.871898 20110 raft_consensus.cc:729] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95e253980cf47a1ab96894a38a3d371, State: Initialized, Role: FOLLOWER
I20260807 06:12:22.871901 19854 heartbeater.cc:503] Master 127.9.133.190:34569 was elected leader, sending a full tablet report...
I20260807 06:12:22.872547 20110 consensus_queue.cc:260] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:22.873462 20113 tablet_bootstrap.cc:492] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: No bootstrap required, opened a new log
I20260807 06:12:22.873720 20113 ts_tablet_manager.cc:1397] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260807 06:12:22.873886 20110 ts_tablet_manager.cc:1428] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260807 06:12:22.875509 20064 heartbeater.cc:503] Master 127.9.133.190:34569 was elected leader, sending a full tablet report...
I20260807 06:12:22.880318 20113 raft_consensus.cc:348] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:22.880617 20113 raft_consensus.cc:374] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:22.880692 20113 raft_consensus.cc:729] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8e26a4d040484fbb0967275acfa7ab, State: Initialized, Role: FOLLOWER
I20260807 06:12:22.881192 20113 consensus_queue.cc:260] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:22.882019 19654 heartbeater.cc:503] Master 127.9.133.190:34569 was elected leader, sending a full tablet report...
I20260807 06:12:22.882421 20113 ts_tablet_manager.cc:1428] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
W20260807 06:12:22.984366 19657 tablet.cc:2369] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:22.997969 19856 tablet.cc:2369] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:23.009812 20067 tablet.cc:2369] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:12:23.016204 20121 raft_consensus.cc:482] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:23.016517 20121 raft_consensus.cc:504] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:23.018178 20121 leader_election.cc:288] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435), fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:23.024153 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.018797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:50291 (local address 127.9.133.130:46105)
0807 06:12:23.018944 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.018949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.018968 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:12:23.019072 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.019075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.019139 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.019233 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:23.019694 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.020600 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.022572 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.023297 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.023416 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.023487 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.023913 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.024020 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:12:23.024152 20103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.018826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:49527 (local address 127.9.133.129:35435)
0807 06:12:23.018982 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.018988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.019025 (+ 37us) server_negotiation.cc:407] Connection header received
0807 06:12:23.019089 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.019092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.019141 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.019233 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:23.019693 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.020591 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.022572 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.023266 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.023437 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.023487 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.023869 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.023980 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:12:23.024130 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.018525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:35435 (local address 127.9.133.131:49527)
0807 06:12:23.018788 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:23.018800 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:23.018866 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:23.019265 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:23.019269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:23.019304 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:23.019524 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:23.019532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.020746 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.020751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:23.022405 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:23.022411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.023439 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.023444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:23.023693 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.023709 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 06:12:23.023801 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:12:23.023923 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":345,"spinlock_wait_cycles":7552}
I20260807 06:12:23.024130 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.018515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:46105 (local address 127.9.133.131:50291)
0807 06:12:23.018680 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:23.018699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:23.018826 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:23.019265 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:23.019269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:23.019302 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:23.019524 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:23.019532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.020747 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.020751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:23.022394 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:23.022407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.023439 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.023444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:23.023650 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.023710 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 06:12:23.023801 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 06:12:23.023940 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":280,"spinlock_wait_cycles":6656}
I20260807 06:12:23.025075 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba3c917c833415086998b16bfad039a" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6649c6dea548c98d4bedc1b73d852e" is_pre_election: true
I20260807 06:12:23.025370 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba3c917c833415086998b16bfad039a" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c8e26a4d040484fbb0967275acfa7ab" is_pre_election: true
I20260807 06:12:23.025496 19787 raft_consensus.cc:2454] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 0.
I20260807 06:12:23.025758 19588 raft_consensus.cc:2454] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 0.
I20260807 06:12:23.026248 19891 leader_election.cc:302] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a95e253980cf47a1ab96894a38a3d371, fb6649c6dea548c98d4bedc1b73d852e; no voters:
I20260807 06:12:23.026626 20121 raft_consensus.cc:2781] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:23.026760 20121 raft_consensus.cc:482] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:23.026837 20121 raft_consensus.cc:3037] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:23.028275 20121 raft_consensus.cc:504] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:23.028795 20121 leader_election.cc:288] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 election: Requested vote from peers 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435), fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:23.029275 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba3c917c833415086998b16bfad039a" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c8e26a4d040484fbb0967275acfa7ab"
I20260807 06:12:23.029304 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba3c917c833415086998b16bfad039a" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6649c6dea548c98d4bedc1b73d852e"
I20260807 06:12:23.029448 19588 raft_consensus.cc:3037] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:23.029490 19787 raft_consensus.cc:3037] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:23.030804 19588 raft_consensus.cc:2454] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 1.
I20260807 06:12:23.031071 19787 raft_consensus.cc:2454] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 1.
I20260807 06:12:23.031219 19893 leader_election.cc:302] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c8e26a4d040484fbb0967275acfa7ab, a95e253980cf47a1ab96894a38a3d371; no voters:
I20260807 06:12:23.031462 20121 raft_consensus.cc:2781] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:23.031728 20121 raft_consensus.cc:686] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 1 LEADER]: Becoming Leader. State: Replica: a95e253980cf47a1ab96894a38a3d371, State: Running, Role: LEADER
I20260807 06:12:23.032264 20121 consensus_queue.cc:237] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } }
I20260807 06:12:23.036700 18288 catalog_manager.cc:5495] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 reported cstate change: term changed from 0 to 1, leader changed from <none> to a95e253980cf47a1ab96894a38a3d371 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "a95e253980cf47a1ab96894a38a3d371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } health_report { overall_health: UNKNOWN } } }
I20260807 06:12:23.087939 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.083292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.9.133.131:39479)
0807 06:12:23.083486 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.083490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.083510 (+ 20us) server_negotiation.cc:407] Connection header received
0807 06:12:23.083624 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.083627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.083702 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.083847 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:23.084179 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.084785 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.085761 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.086133 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.086277 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.086711 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:23.087452 (+ 741us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:23.087568 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.087682 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.087752 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 06:12:23.095978 19588 raft_consensus.cc:1261] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 1 FOLLOWER]: Refusing update from remote peer a95e253980cf47a1ab96894a38a3d371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:23.095983 19787 raft_consensus.cc:1261] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Refusing update from remote peer a95e253980cf47a1ab96894a38a3d371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:23.097056 20125 consensus_queue.cc:1035] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:23.097421 20121 consensus_queue.cc:1035] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:23.109797 20131 mvcc.cc:205] Tried to move back new op lower bound from 7315796554110816256 to 7315796553863806976. Current Snapshot: MvccSnapshot[applied={T|T < 7315796554110816256}]
I20260807 06:12:23.115017 20129 mvcc.cc:205] Tried to move back new op lower bound from 7315796554110816256 to 7315796553863806976. Current Snapshot: MvccSnapshot[applied={T|T < 7315796554110816256}]
I20260807 06:12:23.125233 18288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:23.125481 18288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:23.131255 18288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8896cbe9e2a74deb83262b48f4d99d2d: Sending DeleteTablet for 3 replicas of tablet 5ba3c917c833415086998b16bfad039a
I20260807 06:12:23.132397 19557 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba3c917c833415086998b16bfad039a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:23 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:45546
I20260807 06:12:23.132777 19754 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba3c917c833415086998b16bfad039a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:23 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:57950
I20260807 06:12:23.133378 19958 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba3c917c833415086998b16bfad039a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:23 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:57072
I20260807 06:12:23.133977 20146 tablet_replica.cc:330] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: stopping tablet replica
I20260807 06:12:23.134135 20144 tablet_replica.cc:330] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: stopping tablet replica
I20260807 06:12:23.134421 20146 raft_consensus.cc:2229] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:23.134537 20144 raft_consensus.cc:2229] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:23.135021 20146 raft_consensus.cc:2258] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:23.135025 20144 raft_consensus.cc:2258] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:23.135488 20143 tablet_replica.cc:330] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: stopping tablet replica
I20260807 06:12:23.135838 20143 raft_consensus.cc:2229] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:23.136183 20143 raft_consensus.cc:2258] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:23.136857 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 18232
I20260807 06:12:23.137444 20144 ts_tablet_manager.cc:1905] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:23.140318 20144 ts_tablet_manager.cc:1918] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:23.140409 20144 log.cc:1196] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/5ba3c917c833415086998b16bfad039a
I20260807 06:12:23.140702 20144 ts_tablet_manager.cc:1939] T 5ba3c917c833415086998b16bfad039a P fb6649c6dea548c98d4bedc1b73d852e: Deleting consensus metadata
I20260807 06:12:23.143780 20146 ts_tablet_manager.cc:1905] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:23.146013 20143 ts_tablet_manager.cc:1905] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:23.147223 20146 ts_tablet_manager.cc:1918] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:23.147342 20146 log.cc:1196] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/5ba3c917c833415086998b16bfad039a
I20260807 06:12:23.147746 20146 ts_tablet_manager.cc:1939] T 5ba3c917c833415086998b16bfad039a P a95e253980cf47a1ab96894a38a3d371: Deleting consensus metadata
I20260807 06:12:23.149291 20143 ts_tablet_manager.cc:1918] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:23.149420 20143 log.cc:1196] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/5ba3c917c833415086998b16bfad039a
I20260807 06:12:23.149765 20143 ts_tablet_manager.cc:1939] T 5ba3c917c833415086998b16bfad039a P 0c8e26a4d040484fbb0967275acfa7ab: Deleting consensus metadata
I20260807 06:12:23.158066 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.135684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60846 (local address 127.9.133.189:38265)
0807 06:12:23.135986 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.135992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.136709 (+ 717us) server_negotiation.cc:407] Connection header received
0807 06:12:23.136920 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.136926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.136999 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.137063 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:23.146833 (+ 9770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.148122 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.149880 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.150375 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.150490 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.153504 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:23.153509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:23.153512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:23.153550 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:23.155816 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.156431 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.156435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.156437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.156548 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.156921 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.156924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.156926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.157184 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:23.157317 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.157686 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.157847 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
W20260807 06:12:23.160562 19687 connection.cc:535] server connection from 127.0.0.1:57950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:12:23.160567 19490 connection.cc:535] server connection from 127.0.0.1:45546 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:12:23.161217 19893 connection.cc:535] server connection from 127.0.0.1:57072 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:12:23.169582 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.135818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.9.133.188:37415)
0807 06:12:23.136093 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.136097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.158624 (+ 22527us) server_negotiation.cc:407] Connection header received
0807 06:12:23.158875 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.158882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.158974 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.159078 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:23.160316 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.161240 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.162739 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.163158 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.163267 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.165418 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:23.165423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:23.165426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:23.165462 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:23.167503 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.168034 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.168038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.168040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.168134 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.168445 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.168448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.168450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.168685 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:23.168799 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.169198 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.169382 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260807 06:12:23.177762 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.166366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.9.133.188:37415)
0807 06:12:23.166524 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.166529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.167200 (+ 671us) server_negotiation.cc:407] Connection header received
0807 06:12:23.167426 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.167430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.167493 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.167581 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:23.168641 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.169520 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.171066 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.171437 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.171531 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.173670 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:23.173675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:23.173678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:23.173706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:23.175723 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.176210 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.176217 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.176220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.176346 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.176665 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.176669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.176671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.176951 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:23.177124 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.177354 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.177504 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:12:23.179724 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:23.166448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.9.133.189:38265)
0807 06:12:23.166599 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:23.166612 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:23.167621 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 06:12:23.167825 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:23.167830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:23.168023 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:23.168099 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:23.169066 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.169958 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.172625 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:23.173193 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:23.173370 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:23.175053 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:23.175060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:23.175064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:23.175100 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:23.177112 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.177695 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.177699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.177701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.177811 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:23.178531 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:23.178535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:23.178537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:23.178786 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:23.178892 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:23.179334 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 06:12:23.179494 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:12:24.115558 20159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.114833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:40623)
0807 06:12:24.115238 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.115383 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":160,"threads_started":1}
I20260807 06:12:24.116297 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.115581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:43045)
0807 06:12:24.116049 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.116169 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":88,"threads_started":1}
W20260807 06:12:24.116317 20064 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:34569 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.116703 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.115950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:50375)
0807 06:12:24.116487 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.116592 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":81,"threads_started":1}
W20260807 06:12:24.117007 19654 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:34569 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
W20260807 06:12:24.117167 19854 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:34569 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.632186 20165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8896cbe9e2a74deb83262b48f4d99d2d)
I20260807 06:12:24.632602 20165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:24.632675 20166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8896cbe9e2a74deb83262b48f4d99d2d)
I20260807 06:12:24.632961 20166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:24.634054 20166 leader_election.cc:288] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569), 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189:38265)
I20260807 06:12:24.634292 20165 leader_election.cc:288] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569), 122a8a0352a54dbea9688082e3735d25 (127.9.133.188:37415)
I20260807 06:12:24.634918 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.634252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49664)
0807 06:12:24.634646 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.634760 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":74,"threads_started":1}
I20260807 06:12:24.635119 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.634252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49662)
0807 06:12:24.634891 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.634988 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":93,"threads_started":1}
W20260807 06:12:24.635171 19106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
W20260807 06:12:24.635293 19382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.636157 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.635894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49672)
0807 06:12:24.636038 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.636088 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 06:12:24.636265 19106 leader_election.cc:334] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.636607 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.636378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49682)
0807 06:12:24.636470 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.636529 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 06:12:24.636739 19382 leader_election.cc:334] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.639160 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.634252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37415 (local address 127.0.0.1:36740)
0807 06:12:24.634682 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.634696 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:24.634822 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:24.635365 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:24.635369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:24.635402 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:24.635565 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:24.635575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.636777 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.636781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:24.638014 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:24.638022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.638704 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.638708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:24.638797 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:24.638828 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:12:24.638925 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 06:12:24.639007 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":211,"thread_start_us":78,"threads_started":1}
I20260807 06:12:24.639178 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.634517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.9.133.188:37415)
0807 06:12:24.635080 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:24.635085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:24.635100 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:12:24.635157 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:24.635163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:24.635225 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:24.635689 (+ 464us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:24.635820 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.636603 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.638134 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.638574 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.638699 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:24.638768 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:24.638943 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 06:12:24.639075 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":112,"threads_started":1}
I20260807 06:12:24.639876 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.634624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:38265 (local address 127.0.0.1:60870)
0807 06:12:24.635743 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.635758 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:24.635905 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:24.636522 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:24.636525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:24.636543 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:24.636648 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:24.636657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.637757 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.637760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:24.638612 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:24.638621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.639462 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.639465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:24.639562 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:24.639585 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:12:24.639683 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:12:24.639754 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":61,"threads_started":1}
I20260807 06:12:24.639902 20175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.634815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60870 (local address 127.9.133.189:38265)
0807 06:12:24.635459 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:24.635465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:24.635835 (+ 370us) server_negotiation.cc:407] Connection header received
0807 06:12:24.636030 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:24.636034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:24.636081 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:24.636149 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:24.636788 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.637641 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.638740 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:24.639336 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:24.639460 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:24.639516 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:24.639698 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 06:12:24.639772 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":489,"thread_start_us":81,"threads_started":1}
I20260807 06:12:24.639918 19414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65e2a832f26d4cbc89f10e700713e982" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "122a8a0352a54dbea9688082e3735d25" is_pre_election: true
I20260807 06:12:24.640223 19414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65e2a832f26d4cbc89f10e700713e982 in term 1.
I20260807 06:12:24.640367 19145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "122a8a0352a54dbea9688082e3735d25" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65e2a832f26d4cbc89f10e700713e982" is_pre_election: true
I20260807 06:12:24.640596 19145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 122a8a0352a54dbea9688082e3735d25 in term 1.
I20260807 06:12:24.640631 19111 leader_election.cc:302] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 122a8a0352a54dbea9688082e3735d25, 65e2a832f26d4cbc89f10e700713e982; no voters: 8896cbe9e2a74deb83262b48f4d99d2d
I20260807 06:12:24.640873 20165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 06:12:24.640934 20165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8896cbe9e2a74deb83262b48f4d99d2d)
I20260807 06:12:24.640982 20165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:12:24.641258 19382 leader_election.cc:302] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 122a8a0352a54dbea9688082e3735d25, 65e2a832f26d4cbc89f10e700713e982; no voters: 8896cbe9e2a74deb83262b48f4d99d2d
I20260807 06:12:24.641476 20166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 06:12:24.641538 20166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8896cbe9e2a74deb83262b48f4d99d2d)
I20260807 06:12:24.641579 20166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:12:24.642403 20165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:24.642771 20166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:24.642841 20165 leader_election.cc:288] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 2 election: Requested vote from peers 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569), 122a8a0352a54dbea9688082e3735d25 (127.9.133.188:37415)
I20260807 06:12:24.643164 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.642937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49692)
0807 06:12:24.643039 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.643098 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 06:12:24.643180 20166 leader_election.cc:288] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [CANDIDATE]: Term 2 election: Requested vote from peers 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569), 65e2a832f26d4cbc89f10e700713e982 (127.9.133.189:38265)
I20260807 06:12:24.643381 19414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65e2a832f26d4cbc89f10e700713e982" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "122a8a0352a54dbea9688082e3735d25"
I20260807 06:12:24.643580 19414 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 65e2a832f26d4cbc89f10e700713e982 in current term 2: Already voted for candidate 122a8a0352a54dbea9688082e3735d25 in this term.
I20260807 06:12:24.643736 19145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "122a8a0352a54dbea9688082e3735d25" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65e2a832f26d4cbc89f10e700713e982"
I20260807 06:12:24.643877 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.643373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49694)
0807 06:12:24.643741 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.643803 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307}
I20260807 06:12:24.643921 19145 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 122a8a0352a54dbea9688082e3735d25 in current term 2: Already voted for candidate 65e2a832f26d4cbc89f10e700713e982 in this term.
I20260807 06:12:24.643990 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.643639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49704)
0807 06:12:24.643805 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.643881 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 06:12:24.644085 19106 leader_election.cc:334] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.644153 19106 leader_election.cc:302] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65e2a832f26d4cbc89f10e700713e982; no voters: 122a8a0352a54dbea9688082e3735d25, 8896cbe9e2a74deb83262b48f4d99d2d
I20260807 06:12:24.644296 20165 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 06:12:24.644500 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:24.644281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49716)
0807 06:12:24.644382 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:24.644439 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 06:12:24.644611 19382 leader_election.cc:334] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:24.644682 19382 leader_election.cc:302] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 122a8a0352a54dbea9688082e3735d25; no voters: 65e2a832f26d4cbc89f10e700713e982, 8896cbe9e2a74deb83262b48f4d99d2d
I20260807 06:12:24.644793 20166 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 06:12:25.118011 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:25.117080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:53083)
0807 06:12:25.117729 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:25.117830 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":168,"threads_started":1}
I20260807 06:12:25.118224 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:25.117716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:40955)
0807 06:12:25.118023 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:25.118097 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":102,"threads_started":1}
I20260807 06:12:25.119182 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:25.117885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:40655)
0807 06:12:25.118940 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:25.119053 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":144,"threads_started":1}
I20260807 06:12:26.119963 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.119251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:37123)
0807 06:12:26.119656 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.119752 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":177,"threads_started":1}
I20260807 06:12:26.120568 20184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.119895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:56099)
0807 06:12:26.120343 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.120420 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":84,"threads_started":1}
I20260807 06:12:26.121109 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.120655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:37833)
0807 06:12:26.120925 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.120998 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260807 06:12:26.570777 20186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:26.571079 20186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:26.571815 20186 leader_election.cc:288] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569), 122a8a0352a54dbea9688082e3735d25 (127.9.133.188:37415)
I20260807 06:12:26.572564 19414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65e2a832f26d4cbc89f10e700713e982" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "122a8a0352a54dbea9688082e3735d25" is_pre_election: true
I20260807 06:12:26.572710 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.572070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49972)
0807 06:12:26.572452 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.572592 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":74,"threads_started":1}
I20260807 06:12:26.572801 19414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65e2a832f26d4cbc89f10e700713e982 in term 2.
I20260807 06:12:26.573266 19111 leader_election.cc:302] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 122a8a0352a54dbea9688082e3735d25, 65e2a832f26d4cbc89f10e700713e982; no voters:
I20260807 06:12:26.573484 20186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 06:12:26.573589 20186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:26.573658 20186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:12:26.574055 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.573770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49984)
0807 06:12:26.573890 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.573951 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 06:12:26.574229 19106 leader_election.cc:334] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:26.575403 20186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:26.575814 20186 leader_election.cc:288] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 3 election: Requested vote from peers 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569), 122a8a0352a54dbea9688082e3735d25 (127.9.133.188:37415)
I20260807 06:12:26.576254 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.575908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49986)
0807 06:12:26.576106 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.576156 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 06:12:26.576370 19414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65e2a832f26d4cbc89f10e700713e982" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "122a8a0352a54dbea9688082e3735d25"
I20260807 06:12:26.576574 19414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:12:26.577047 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.576777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:49996)
0807 06:12:26.576898 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.576953 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 06:12:26.577162 19106 leader_election.cc:334] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
I20260807 06:12:26.578027 19414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65e2a832f26d4cbc89f10e700713e982 in term 3.
I20260807 06:12:26.578408 19111 leader_election.cc:302] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 122a8a0352a54dbea9688082e3735d25, 65e2a832f26d4cbc89f10e700713e982; no voters: 8896cbe9e2a74deb83262b48f4d99d2d
I20260807 06:12:26.578649 20186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:12:26.578859 20186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [term 3 LEADER]: Becoming Leader. State: Replica: 65e2a832f26d4cbc89f10e700713e982, State: Running, Role: LEADER
I20260807 06:12:26.579241 20186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } }
I20260807 06:12:26.581116 20190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65e2a832f26d4cbc89f10e700713e982. Latest consensus state: current_term: 3 leader_uuid: "65e2a832f26d4cbc89f10e700713e982" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:26.581287 20190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:26.581650 20194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:26.584928 20194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:26.585347 20194 catalog_manager.cc:1259] Loaded cluster ID: 30ab31c3798842e1b005d70a8070818e
I20260807 06:12:26.585397 20194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:26.586009 20194 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:26.586369 20194 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Loaded TSK: 0
I20260807 06:12:26.587280 20194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:26.614570 19131 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:26.618695 19131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:26.622829 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.622534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:50010)
0807 06:12:26.622678 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.622738 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 06:12:26.623064 19414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [term 3 FOLLOWER]: Refusing update from remote peer 65e2a832f26d4cbc89f10e700713e982: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 06:12:26.623698 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.623438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.0.0.1:50016)
0807 06:12:26.623538 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.623602 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 06:12:26.623755 20186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [LEADER]: Connected to new peer: Peer: permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 06:12:26.624022 19106 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 -> Peer 8896cbe9e2a74deb83262b48f4d99d2d (127.9.133.190:34569): Couldn't send request to peer 8896cbe9e2a74deb83262b48f4d99d2d. Status: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 06:12:26.625633 20197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65e2a832f26d4cbc89f10e700713e982. Latest consensus state: current_term: 3 leader_uuid: "65e2a832f26d4cbc89f10e700713e982" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:26.625788 20197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:26.627351 20190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "65e2a832f26d4cbc89f10e700713e982" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:26.627489 20190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:26.627817 20197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "65e2a832f26d4cbc89f10e700713e982" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8896cbe9e2a74deb83262b48f4d99d2d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 34569 } } peers { permanent_uuid: "65e2a832f26d4cbc89f10e700713e982" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 38265 } attrs { promote: false } } peers { permanent_uuid: "122a8a0352a54dbea9688082e3735d25" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37415 } attrs { promote: false } } }
I20260807 06:12:26.628019 20197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122a8a0352a54dbea9688082e3735d25 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:12:26.640008 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.634878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:35435 (local address 127.0.0.1:45554)
0807 06:12:26.635064 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.635085 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:26.635191 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:26.636250 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:26.636256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:26.636279 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:26.636415 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:26.636423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.637588 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.637594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:26.638504 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:26.638538 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.639499 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.639504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:26.639612 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.639639 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 06:12:26.639745 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:12:26.639834 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":1}
I20260807 06:12:26.640018 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.635460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.9.133.129:35435)
0807 06:12:26.635812 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:26.635819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:26.635838 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:12:26.635913 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:26.635919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:26.636048 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:26.636235 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:26.636554 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.637478 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.638651 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.639369 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.639496 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.639584 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:26.639769 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 06:12:26.639864 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":105,"threads_started":1}
I20260807 06:12:26.640753 19557 tablet_service.cc:1467] Processing CreateTablet for tablet d781965cb5004248b4ce1858e3262313 (DEFAULT_TABLE table=test-table [id=1188531e3707465ea9b2020c2b1b6bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:26.641197 19557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d781965cb5004248b4ce1858e3262313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:26.641772 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.636160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:39479 (local address 127.0.0.1:57088)
0807 06:12:26.636887 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.636903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:26.637001 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:26.637400 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:26.637403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:26.637419 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:26.637538 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:26.637545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.638589 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.638594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:26.639850 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:26.639861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.641264 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.641268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:26.641365 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.641409 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 06:12:26.641484 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:12:26.641576 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":91,"threads_started":1}
I20260807 06:12:26.641978 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.635792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:46105 (local address 127.0.0.1:57952)
0807 06:12:26.636804 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.636822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:26.636925 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:26.637227 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:26.637230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:26.638032 (+ 802us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:26.638152 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:26.638158 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.639231 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.639235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:26.640436 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:26.640447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.641500 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.641504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:26.641610 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.641632 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 06:12:26.641682 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 06:12:26.641746 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":1,"thread_start_us":133,"threads_started":1}
I20260807 06:12:26.642302 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.636014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.9.133.130:46105)
0807 06:12:26.636908 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:26.636915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:26.636940 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:12:26.637046 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:26.637051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:26.637108 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:26.637223 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:26.638295 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.639105 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.640593 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.641356 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.641875 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.641989 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:26.642034 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 06:12:26.642137 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":102,"threads_started":1}
I20260807 06:12:26.642387 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.636463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.9.133.131:39479)
0807 06:12:26.637152 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:26.637157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:26.637173 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:12:26.637238 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:26.637242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:26.637295 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:26.637392 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:26.637689 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.638458 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.640011 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.641108 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.641987 (+ 879us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.642069 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:26.642110 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 06:12:26.642209 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":105,"threads_started":1}
I20260807 06:12:26.644294 20221 tablet_bootstrap.cc:492] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab: Bootstrap starting.
I20260807 06:12:26.644836 19754 tablet_service.cc:1467] Processing CreateTablet for tablet d781965cb5004248b4ce1858e3262313 (DEFAULT_TABLE table=test-table [id=1188531e3707465ea9b2020c2b1b6bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:26.644855 19958 tablet_service.cc:1467] Processing CreateTablet for tablet d781965cb5004248b4ce1858e3262313 (DEFAULT_TABLE table=test-table [id=1188531e3707465ea9b2020c2b1b6bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:26.645220 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d781965cb5004248b4ce1858e3262313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:26.645253 19958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d781965cb5004248b4ce1858e3262313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:26.645756 20221 tablet_bootstrap.cc:654] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:26.647485 20221 tablet_bootstrap.cc:492] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab: No bootstrap required, opened a new log
I20260807 06:12:26.647627 20221 ts_tablet_manager.cc:1397] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:26.648288 20221 raft_consensus.cc:348] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.648460 20221 raft_consensus.cc:374] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:26.648543 20225 tablet_bootstrap.cc:492] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Bootstrap starting.
I20260807 06:12:26.648550 20221 raft_consensus.cc:729] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8e26a4d040484fbb0967275acfa7ab, State: Initialized, Role: FOLLOWER
I20260807 06:12:26.648772 20221 consensus_queue.cc:260] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.649230 20221 ts_tablet_manager.cc:1428] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:12:26.649246 20224 tablet_bootstrap.cc:492] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Bootstrap starting.
I20260807 06:12:26.649686 20225 tablet_bootstrap.cc:654] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:26.650898 20224 tablet_bootstrap.cc:654] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:26.651427 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.650856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:42773)
0807 06:12:26.651179 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.651261 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":111,"threads_started":1}
I20260807 06:12:26.652498 20224 tablet_bootstrap.cc:492] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: No bootstrap required, opened a new log
I20260807 06:12:26.652613 20224 ts_tablet_manager.cc:1397] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:26.653149 20224 raft_consensus.cc:348] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.653290 20224 raft_consensus.cc:374] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:26.653328 19655 heartbeater.cc:503] Master 127.9.133.189:38265 was elected leader, sending a full tablet report...
I20260807 06:12:26.653338 20224 raft_consensus.cc:729] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6649c6dea548c98d4bedc1b73d852e, State: Initialized, Role: FOLLOWER
I20260807 06:12:26.653508 20224 consensus_queue.cc:260] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.653920 20224 ts_tablet_manager.cc:1428] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 06:12:26.654277 20225 tablet_bootstrap.cc:492] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: No bootstrap required, opened a new log
I20260807 06:12:26.654400 20225 ts_tablet_manager.cc:1397] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 06:12:26.654906 20225 raft_consensus.cc:348] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.655057 20225 raft_consensus.cc:374] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:26.655095 20225 raft_consensus.cc:729] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95e253980cf47a1ab96894a38a3d371, State: Initialized, Role: FOLLOWER
I20260807 06:12:26.655236 20225 consensus_queue.cc:260] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.655643 20225 ts_tablet_manager.cc:1428] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 06:12:26.657385 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.656341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:42857)
0807 06:12:26.657179 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.657257 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":143,"threads_started":1}
I20260807 06:12:26.658186 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.655800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:57239)
0807 06:12:26.657981 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.658049 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2079,"thread_start_us":122,"threads_started":1}
I20260807 06:12:26.658370 19855 heartbeater.cc:503] Master 127.9.133.189:38265 was elected leader, sending a full tablet report...
I20260807 06:12:26.659086 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.658768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:56423)
0807 06:12:26.658925 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.658984 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 06:12:26.659570 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.658650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:60729)
0807 06:12:26.659412 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.659502 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":686}
I20260807 06:12:26.659931 20065 heartbeater.cc:503] Master 127.9.133.189:38265 was elected leader, sending a full tablet report...
I20260807 06:12:26.779345 20233 raft_consensus.cc:482] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:26.779588 20233 raft_consensus.cc:504] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.780192 20233 leader_election.cc:288] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435), fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:26.780923 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d781965cb5004248b4ce1858e3262313" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c8e26a4d040484fbb0967275acfa7ab" is_pre_election: true
I20260807 06:12:26.780923 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d781965cb5004248b4ce1858e3262313" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6649c6dea548c98d4bedc1b73d852e" is_pre_election: true
I20260807 06:12:26.781144 19787 raft_consensus.cc:2454] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 0.
I20260807 06:12:26.781144 19588 raft_consensus.cc:2454] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 0.
I20260807 06:12:26.781613 19893 leader_election.cc:302] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c8e26a4d040484fbb0967275acfa7ab, a95e253980cf47a1ab96894a38a3d371, fb6649c6dea548c98d4bedc1b73d852e; no voters:
I20260807 06:12:26.781868 20233 raft_consensus.cc:2781] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:26.781982 20233 raft_consensus.cc:482] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:26.782040 20233 raft_consensus.cc:3037] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:26.783746 20233 raft_consensus.cc:504] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.784183 20233 leader_election.cc:288] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 election: Requested vote from peers 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435), fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105)
I20260807 06:12:26.784754 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d781965cb5004248b4ce1858e3262313" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6649c6dea548c98d4bedc1b73d852e"
I20260807 06:12:26.784754 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d781965cb5004248b4ce1858e3262313" candidate_uuid: "a95e253980cf47a1ab96894a38a3d371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c8e26a4d040484fbb0967275acfa7ab"
I20260807 06:12:26.784986 19787 raft_consensus.cc:3037] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:26.784986 19588 raft_consensus.cc:3037] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:26.786502 19787 raft_consensus.cc:2454] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 1.
I20260807 06:12:26.786504 19588 raft_consensus.cc:2454] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95e253980cf47a1ab96894a38a3d371 in term 1.
I20260807 06:12:26.786899 19891 leader_election.cc:302] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a95e253980cf47a1ab96894a38a3d371, fb6649c6dea548c98d4bedc1b73d852e; no voters:
I20260807 06:12:26.787153 20233 raft_consensus.cc:2781] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:26.787478 20233 raft_consensus.cc:686] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 1 LEADER]: Becoming Leader. State: Replica: a95e253980cf47a1ab96894a38a3d371, State: Running, Role: LEADER
I20260807 06:12:26.787686 20233 consensus_queue.cc:237] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } }
I20260807 06:12:26.789209 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.788816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:38887)
0807 06:12:26.789023 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.789099 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 06:12:26.790026 19129 catalog_manager.cc:5495] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 reported cstate change: term changed from 0 to 1, leader changed from <none> to a95e253980cf47a1ab96894a38a3d371 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "a95e253980cf47a1ab96894a38a3d371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95e253980cf47a1ab96894a38a3d371" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39479 } health_report { overall_health: HEALTHY } } }
I20260807 06:12:26.835634 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.832084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.9.133.131:39479)
0807 06:12:26.832237 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:26.832241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:26.832255 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:26.832315 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:26.832317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:26.832370 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:26.832456 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:26.832800 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.833412 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.834360 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:26.834579 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:26.834668 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:26.834956 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:26.835107 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:26.835177 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:26.835418 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 06:12:26.835479 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:12:26.838636 19787 raft_consensus.cc:1261] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Refusing update from remote peer a95e253980cf47a1ab96894a38a3d371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:26.838657 19588 raft_consensus.cc:1261] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab [term 1 FOLLOWER]: Refusing update from remote peer a95e253980cf47a1ab96894a38a3d371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:26.839244 20233 consensus_queue.cc:1035] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6649c6dea548c98d4bedc1b73d852e" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 46105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:26.839499 20236 consensus_queue.cc:1035] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c8e26a4d040484fbb0967275acfa7ab" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 35435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:26.841534 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.841177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:37869)
0807 06:12:26.841348 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.841429 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 06:12:26.841931 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.841539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:41949)
0807 06:12:26.841744 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.841813 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 06:12:26.845928 20239 mvcc.cc:205] Tried to move back new op lower bound from 7315796569445466112 to 7315796569244688384. Current Snapshot: MvccSnapshot[applied={T|T < 7315796569445466112}]
I20260807 06:12:26.846519 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.846224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.129:52723)
0807 06:12:26.846397 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.846448 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 06:12:26.846871 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.846224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:60465)
0807 06:12:26.846718 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.846778 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414}
I20260807 06:12:26.855270 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.854181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.130:47057)
0807 06:12:26.855091 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.855169 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":792}
I20260807 06:12:26.855865 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.854825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:34569 (local address 127.9.133.131:44641)
0807 06:12:26.855707 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.855768 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":784}
I20260807 06:12:26.857254 19131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:26.857450 19131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:26.862869 19131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65e2a832f26d4cbc89f10e700713e982: Sending DeleteTablet for 3 replicas of tablet d781965cb5004248b4ce1858e3262313
I20260807 06:12:26.863899 19557 tablet_service.cc:1514] Processing DeleteTablet for tablet d781965cb5004248b4ce1858e3262313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:26 UTC) from {username='oryx', principal='oryx/127.9.133.189@KRBTEST.COM'} at 127.0.0.1:45554
I20260807 06:12:26.863919 19958 tablet_service.cc:1514] Processing DeleteTablet for tablet d781965cb5004248b4ce1858e3262313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:26 UTC) from {username='oryx', principal='oryx/127.9.133.189@KRBTEST.COM'} at 127.0.0.1:57088
I20260807 06:12:26.864053 19754 tablet_service.cc:1514] Processing DeleteTablet for tablet d781965cb5004248b4ce1858e3262313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:26 UTC) from {username='oryx', principal='oryx/127.9.133.189@KRBTEST.COM'} at 127.0.0.1:57952
I20260807 06:12:26.864576 20253 tablet_replica.cc:330] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: stopping tablet replica
I20260807 06:12:26.864600 20250 tablet_replica.cc:330] T d781965cb5004248b4ce1858e3262313 P 0c8e26a4d040484fbb0967275acfa7ab: stopping tablet replica
I20260807 06:12:26.864764 20253 raft_consensus.cc:2229] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:26.864902 20251 tablet_replica.cc:330] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: stopping tablet replica
I20260807 06:12:26.864939 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 19334
I20260807 06:12:26.864964 20253 raft_consensus.cc:2258] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:26.865108 20251 raft_consensus.cc:2229] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:26.865413 20251 raft_consensus.cc:2258] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:26.866143 20253 ts_tablet_manager.cc:1905] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:26.870919 20253 ts_tablet_manager.cc:1918] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:26.871091 20253 log.cc:1196] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/d781965cb5004248b4ce1858e3262313
I20260807 06:12:26.871436 20253 ts_tablet_manager.cc:1939] T d781965cb5004248b4ce1858e3262313 P fb6649c6dea548c98d4bedc1b73d852e: Deleting consensus metadata
I20260807 06:12:26.873538 19109 catalog_manager.cc:4841] TS fb6649c6dea548c98d4bedc1b73d852e (127.9.133.130:46105): tablet d781965cb5004248b4ce1858e3262313 (table test-table [id=1188531e3707465ea9b2020c2b1b6bf0]) successfully deleted
I20260807 06:12:26.874097 20251 ts_tablet_manager.cc:1905] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:26.876927 20251 ts_tablet_manager.cc:1918] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:26.877038 20251 log.cc:1196] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/d781965cb5004248b4ce1858e3262313
I20260807 06:12:26.877364 20251 ts_tablet_manager.cc:1939] T d781965cb5004248b4ce1858e3262313 P a95e253980cf47a1ab96894a38a3d371: Deleting consensus metadata
I20260807 06:12:26.878269 19109 catalog_manager.cc:4841] TS a95e253980cf47a1ab96894a38a3d371 (127.9.133.131:39479): tablet d781965cb5004248b4ce1858e3262313 (table test-table [id=1188531e3707465ea9b2020c2b1b6bf0]) successfully deleted
I20260807 06:12:26.883546 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 19662
I20260807 06:12:26.883749 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:26.883374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:35435 (local address 127.0.0.1:45560)
0807 06:12:26.883541 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:26.883613 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:35435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 06:12:26.884622 19111 catalog_manager.cc:4584] TS 0c8e26a4d040484fbb0967275acfa7ab (127.9.133.129:35435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d781965cb5004248b4ce1858e3262313: Network error: Client connection negotiation failed: client connection to 127.9.133.129:35435: connect: Connection refused (error 111)
I20260807 06:12:26.899302 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 19863
I20260807 06:12:26.913210 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 18926
I20260807 06:12:26.927343 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 19077
2026-08-07T06:12:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (8343 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:27 dist-test-slave-h3ft krb5kdc[20258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:27 dist-test-slave-h3ft krb5kdc[20258](info): set up 2 sockets
Aug 07 06:12:27 dist-test-slave-h3ft krb5kdc[20258](info): commencing operation
krb5kdc: starting...
W20260807 06:12:28.244508 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:28 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab.
Entry for principal oryx/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab.
I20260807 06:12:28.390528 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:41417
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab
--principal=oryx/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:41417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:28.523833 20276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:28.524153 20276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:28.524195 20276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:28.528355 20276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:28.528424 20276 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 06:12:28.528441 20276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:28.528466 20276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:28.528483 20276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:28.528498 20276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:28.534312 20276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:41417
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.190.keytab
--principal=oryx/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:41417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:28.535787 20276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:28.537118 20276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:12:28.547864 20276 server_base.cc:1034] running on GCE node
W20260807 06:12:28.547923 20285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:28.547832 20283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:28.547832 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:28.548835 20276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:28.550235 20276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:28.551440 20276 hybrid_clock.cc:648] HybridClock initialized: now 1786083148551403 us; error 48 us; skew 500 ppm
Aug 07 06:12:28 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083148, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:28.554775 20276 init.cc:377] Logged in from keytab as oryx/127.9.133.190@KRBTEST.COM (short username oryx)
I20260807 06:12:28.555943 20276 webserver.cc:460] Webserver started at http://127.9.133.190:37769/ using document root <none> and password file <none>
I20260807 06:12:28.556596 20276 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:28.556664 20276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:28.556881 20276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:28.559480 20276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "4882b111e035492f8dd4f5bbfc9f3d57"
format_stamp: "Formatted at 2026-08-07 06:12:28 on dist-test-slave-h3ft"
server_key: "8691755474ccd9b9ff1bb0e362c9b8d2"
server_key_iv: "c3eec21b957b0e116053108131e5ce9e"
server_key_version: "encryptionkey@0"
I20260807 06:12:28.560106 20276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "4882b111e035492f8dd4f5bbfc9f3d57"
format_stamp: "Formatted at 2026-08-07 06:12:28 on dist-test-slave-h3ft"
server_key: "8691755474ccd9b9ff1bb0e362c9b8d2"
server_key_iv: "c3eec21b957b0e116053108131e5ce9e"
server_key_version: "encryptionkey@0"
I20260807 06:12:28.564827 20276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:28.567766 20296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:28.569342 20276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:28.569517 20276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "4882b111e035492f8dd4f5bbfc9f3d57"
format_stamp: "Formatted at 2026-08-07 06:12:28 on dist-test-slave-h3ft"
server_key: "8691755474ccd9b9ff1bb0e362c9b8d2"
server_key_iv: "c3eec21b957b0e116053108131e5ce9e"
server_key_version: "encryptionkey@0"
I20260807 06:12:28.569648 20276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:28.598321 20276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:28.602260 20276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:28.602524 20276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:28.612720 20276 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:41417
I20260807 06:12:28.612737 20387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:41417 every 8 connection(s)
I20260807 06:12:28.613916 20276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:28.617566 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:28.619575 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 20276
I20260807 06:12:28.619735 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:28.620096 9750 external_mini_cluster.cc:1428] Setting key acbb5f7e5ee6f393d5319ac948e392f8
I20260807 06:12:28.626317 20389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Bootstrap starting.
I20260807 06:12:28.630000 20389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:28.631088 20389 log.cc:824] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:12:28 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:28.634346 20389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: No bootstrap required, opened a new log
I20260807 06:12:28.639683 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:28.621895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.9.133.190:41417)
0807 06:12:28.622430 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:28.622456 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:28.622512 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:12:28.623555 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:28.623585 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:28.624125 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:28.624576 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:28.625331 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.627047 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.628055 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.628857 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.629073 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:28.632500 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:28.632508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:28.632531 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:28.632568 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:28.635699 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:28.636215 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:28.636223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:28.636231 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:28.636403 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:28.636711 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:28.636717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:28.636720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:28.637386 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:28.637612 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:28.637943 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 06:12:28.638343 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":138,"threads_started":1}
I20260807 06:12:28.639508 20389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 41417 } }
I20260807 06:12:28.639851 20389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:28.639987 20389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4882b111e035492f8dd4f5bbfc9f3d57, State: Initialized, Role: FOLLOWER
I20260807 06:12:28.640568 20389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 41417 } }
I20260807 06:12:28.640780 20389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:28.640885 20389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:28.641044 20389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:28.642643 20389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 41417 } }
I20260807 06:12:28.643173 20389 leader_election.cc:302] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4882b111e035492f8dd4f5bbfc9f3d57; no voters:
I20260807 06:12:28.643596 20389 leader_election.cc:288] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:28.643735 20395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:28.644045 20395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [term 1 LEADER]: Becoming Leader. State: Replica: 4882b111e035492f8dd4f5bbfc9f3d57, State: Running, Role: LEADER
I20260807 06:12:28.644446 20395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 41417 } }
I20260807 06:12:28.645243 20389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:28.646523 20399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4882b111e035492f8dd4f5bbfc9f3d57. Latest consensus state: current_term: 1 leader_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 41417 } } }
I20260807 06:12:28.646688 20399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:28.646647 20397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4882b111e035492f8dd4f5bbfc9f3d57" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 41417 } } }
I20260807 06:12:28.646745 20397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:28.648085 20410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:28.652103 20410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:28.659467 20410 catalog_manager.cc:1347] Generated new cluster ID: 800d245a0dbe42e0a775ed93c929f0d6
I20260807 06:12:28.659552 20410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:28.681568 20410 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:28.682783 20410 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:28.692276 20410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Generated new TSK 0
I20260807 06:12:28.693259 20410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab.
Entry for principal oryx/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab.
I20260807 06:12:28.734831 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:41417
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:28.875334 20424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:28.875684 20424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:28.875761 20424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:28.880064 20424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:28.880141 20424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:28.880187 20424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:12:28.880285 20424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:28.886250 20424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:28.887691 20424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:28.889025 20424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:28.900964 20434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:28.900872 20432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:28.900875 20438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:28.901391 20424 server_base.cc:1034] running on GCE node
I20260807 06:12:28.901881 20424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:28.902657 20424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:28.903885 20424 hybrid_clock.cc:648] HybridClock initialized: now 1786083148903841 us; error 54 us; skew 500 ppm
Aug 07 06:12:28 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083148, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:28.907370 20424 init.cc:377] Logged in from keytab as oryx/127.9.133.129@KRBTEST.COM (short username oryx)
I20260807 06:12:28.908485 20424 webserver.cc:460] Webserver started at http://127.9.133.129:43723/ using document root <none> and password file <none>
I20260807 06:12:28.909165 20424 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:28.909232 20424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:28.909440 20424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:28.912027 20424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "d5e7776bf5684c1cbad259685e719276"
format_stamp: "Formatted at 2026-08-07 06:12:28 on dist-test-slave-h3ft"
server_key: "294a09e1a70ee5dfbb3695161a7d7792"
server_key_iv: "ce64b2d39e95842636892305b8999af0"
server_key_version: "encryptionkey@0"
I20260807 06:12:28.912657 20424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "d5e7776bf5684c1cbad259685e719276"
format_stamp: "Formatted at 2026-08-07 06:12:28 on dist-test-slave-h3ft"
server_key: "294a09e1a70ee5dfbb3695161a7d7792"
server_key_iv: "ce64b2d39e95842636892305b8999af0"
server_key_version: "encryptionkey@0"
I20260807 06:12:28.917243 20424 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:28.920173 20450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:28.921710 20424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:28.921883 20424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "d5e7776bf5684c1cbad259685e719276"
format_stamp: "Formatted at 2026-08-07 06:12:28 on dist-test-slave-h3ft"
server_key: "294a09e1a70ee5dfbb3695161a7d7792"
server_key_iv: "ce64b2d39e95842636892305b8999af0"
server_key_version: "encryptionkey@0"
I20260807 06:12:28.922021 20424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:28.941694 20424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:28.946055 20424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:28.946295 20424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:28.948406 20424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:28.948493 20424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:28.948594 20424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:28.948639 20424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:12:28 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083148, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:28.962000 20424 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:43521
I20260807 06:12:28.962049 20626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:43521 every 8 connection(s)
I20260807 06:12:28.963384 20424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:28.964583 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 20424
I20260807 06:12:28.964766 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:28.965127 9750 external_mini_cluster.cc:1428] Setting key 036023cb8d24cff5911cbf3c30575db8
I20260807 06:12:28.965621 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:28.949395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:35477 (local address 127.9.133.190:41417)
0807 06:12:28.949571 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:28.949575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:28.950875 (+ 1300us) server_negotiation.cc:407] Connection header received
0807 06:12:28.952098 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:28.952101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:28.952182 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:28.952279 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:28.953911 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.954632 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.956045 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.956315 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.956455 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:28.960709 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:28.960713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:28.960715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:28.960749 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:28.962728 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:28.963444 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:28.963447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:28.963449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:28.963598 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:28.964082 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:28.964086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:28.964088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:28.964413 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:28.964548 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:28.965221 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 06:12:28.965397 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:12:28.966741 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:28.950106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.9.133.129:35477)
0807 06:12:28.950688 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:28.950742 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:28.951823 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:28.952664 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:28.952672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:28.953060 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:28.953727 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:28.953744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.954780 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.954789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:28.955909 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:28.955920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.956500 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.956504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:28.956752 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:28.957580 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:28.957598 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:28.960486 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:28.962867 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:28.962874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:28.962892 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:28.963306 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:28.963715 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:28.963721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:28.963726 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:28.963950 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:28.964562 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:28.964572 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:28.964932 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 06:12:28.965147 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 06:12:28.965551 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":157,"threads_started":1}
I20260807 06:12:28.968199 20627 heartbeater.cc:347] Connected to a master server at 127.9.133.190:41417
I20260807 06:12:28.968502 20627 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:28.969238 20627 heartbeater.cc:511] Master 127.9.133.190:41417 requested a full tablet report, sending...
I20260807 06:12:28.971333 20323 ts_manager.cc:194] Registered new tserver with Master: d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521)
I20260807 06:12:28.973099 20323 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:35477
WARNING: no policy specified for oryx/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:28.982337 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:28.973068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.9.133.129:58921)
0807 06:12:28.973538 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:28.973556 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:28.973680 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:28.974082 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:28.974085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:28.974403 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:28.974587 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:28.974600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.975725 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.975732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:28.976718 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:28.976731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.977302 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.977309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:28.977422 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:28.978095 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:28.978118 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:28.978614 (+ 496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:28.980590 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:28.980593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:28.980595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:28.980802 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:28.981163 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:28.981165 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:28.981166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:28.981248 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:28.981685 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:28.981688 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:28.981824 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 06:12:28.981947 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 06:12:28.982113 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":249,"threads_started":1}
I20260807 06:12:28.982378 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:28.973139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:58921 (local address 127.9.133.190:41417)
0807 06:12:28.973279 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:28.973284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:28.973656 (+ 372us) server_negotiation.cc:407] Connection header received
0807 06:12:28.973776 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:28.973781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:28.973843 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:28.973929 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:28.974734 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.975571 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.976865 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:28.977153 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:28.977260 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:28.978768 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:28.978772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:28.978774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:28.978798 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:28.980488 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:28.980934 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:28.980937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:28.980938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:28.981057 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:28.981331 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:28.981334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:28.981335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:28.981569 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:28.981698 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:28.982042 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 06:12:28.982200 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab.
Entry for principal oryx/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab.
I20260807 06:12:29.003720 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:41417
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:29.140798 20638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:29.141155 20638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:29.141207 20638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:29.145550 20638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:29.145630 20638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:29.145673 20638 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:12:29.145711 20638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:29.152168 20638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:29.153687 20638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:29.155108 20638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:29.166677 20644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:29.166714 20647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:29.166769 20645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:29.167263 20638 server_base.cc:1034] running on GCE node
I20260807 06:12:29.167781 20638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:29.168633 20638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:29.169863 20638 hybrid_clock.cc:648] HybridClock initialized: now 1786083149169834 us; error 35 us; skew 500 ppm
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083149, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:29.173584 20638 init.cc:377] Logged in from keytab as oryx/127.9.133.130@KRBTEST.COM (short username oryx)
I20260807 06:12:29.174834 20638 webserver.cc:460] Webserver started at http://127.9.133.130:41137/ using document root <none> and password file <none>
I20260807 06:12:29.175705 20638 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:29.175812 20638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:29.176090 20638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:29.178643 20638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
format_stamp: "Formatted at 2026-08-07 06:12:29 on dist-test-slave-h3ft"
server_key: "1bca7e0fed4a1548b5583ebe992aa335"
server_key_iv: "5ac958cbb97dc89a0231dbbd93517f96"
server_key_version: "encryptionkey@0"
I20260807 06:12:29.179352 20638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
format_stamp: "Formatted at 2026-08-07 06:12:29 on dist-test-slave-h3ft"
server_key: "1bca7e0fed4a1548b5583ebe992aa335"
server_key_iv: "5ac958cbb97dc89a0231dbbd93517f96"
server_key_version: "encryptionkey@0"
I20260807 06:12:29.183949 20638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:29.186746 20658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:29.188230 20638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:12:29.188386 20638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
format_stamp: "Formatted at 2026-08-07 06:12:29 on dist-test-slave-h3ft"
server_key: "1bca7e0fed4a1548b5583ebe992aa335"
server_key_iv: "5ac958cbb97dc89a0231dbbd93517f96"
server_key_version: "encryptionkey@0"
I20260807 06:12:29.188508 20638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:29.205504 20638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:29.209519 20638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:29.209767 20638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:29.211702 20638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:29.211783 20638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:29.211874 20638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:29.211930 20638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083149, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:29.224694 20638 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:38123
I20260807 06:12:29.224788 20823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:38123 every 8 connection(s)
I20260807 06:12:29.226048 20638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:29.228576 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.212610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:56405 (local address 127.9.133.190:41417)
0807 06:12:29.212778 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.212783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.214124 (+ 1341us) server_negotiation.cc:407] Connection header received
0807 06:12:29.215449 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.215455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.215536 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.215613 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:29.217820 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.218819 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.220801 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.221155 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.221280 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.224454 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:29.224459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:29.224462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:29.224504 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:29.226136 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.226841 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.226846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.226848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.226964 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.227405 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.227408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.227409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.227632 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:29.227719 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.228264 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.228424 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 06:12:29.229503 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.213271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.9.133.130:56405)
0807 06:12:29.213933 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.213990 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.215148 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.216162 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.216173 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.216777 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:29.217605 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.217629 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.219055 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.219059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.220644 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.220657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.221333 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.221336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.221496 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.222251 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:29.222276 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:29.224237 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:29.226268 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.226274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.226290 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.226702 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.227098 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.227103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.227105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.227283 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.227728 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:29.227734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:29.228017 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.228183 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.228465 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":151,"threads_started":1}
I20260807 06:12:29.230834 20824 heartbeater.cc:347] Connected to a master server at 127.9.133.190:41417
I20260807 06:12:29.230990 20824 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:29.231671 20824 heartbeater.cc:511] Master 127.9.133.190:41417 requested a full tablet report, sending...
I20260807 06:12:29.233011 20322 ts_manager.cc:194] Registered new tserver with Master: 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123)
I20260807 06:12:29.233922 20322 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:56405
I20260807 06:12:29.234337 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 20638
I20260807 06:12:29.234488 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:29.234889 9750 external_mini_cluster.cc:1428] Setting key 31e05425c7603f629f721494b300891f
I20260807 06:12:29.244017 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.233710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:39069 (local address 127.9.133.190:41417)
0807 06:12:29.233835 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.233839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.234147 (+ 308us) server_negotiation.cc:407] Connection header received
0807 06:12:29.234306 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.234312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.234367 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.234422 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:29.236940 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.237470 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.238709 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.238914 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.238992 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.240337 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:29.240344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:29.240350 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:29.240384 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:29.241992 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.242504 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.242506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.242508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.242584 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.242908 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.242910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.242911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.243207 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:29.243290 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.243763 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.243866 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:12:29.244125 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.233639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.9.133.130:39069)
0807 06:12:29.234034 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.234054 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.234176 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.236395 (+ 2219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.236417 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.236679 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:29.236810 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.236818 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.237616 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.237622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.238590 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.238605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.239048 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.239054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.239183 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.239830 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:29.239842 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:29.240193 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:29.242110 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.242117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.242122 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.242407 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.242676 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.242683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.242688 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.242805 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.243435 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:29.243442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:29.243570 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.243705 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.243868 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab.
Entry for principal oryx/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab.
I20260807 06:12:29.271178 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:41417
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:29.404505 20834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:29.404855 20834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:29.404903 20834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:29.409240 20834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:29.409314 20834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:29.409356 20834 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 06:12:29.409392 20834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:29.416649 20834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.133.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:29.418182 20834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:29.419598 20834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:29.431135 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:29.431191 20844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:29.431166 20841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:29.431337 20834 server_base.cc:1034] running on GCE node
I20260807 06:12:29.431901 20834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:29.432751 20834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:29.433918 20834 hybrid_clock.cc:648] HybridClock initialized: now 1786083149433884 us; error 46 us; skew 500 ppm
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083149, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:29.437179 20834 init.cc:377] Logged in from keytab as oryx/127.9.133.131@KRBTEST.COM (short username oryx)
I20260807 06:12:29.438274 20834 webserver.cc:460] Webserver started at http://127.9.133.131:44025/ using document root <none> and password file <none>
I20260807 06:12:29.438966 20834 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:29.439119 20834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:29.439404 20834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:29.441846 20834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "9e13573b085646f9a34a548444aae625"
format_stamp: "Formatted at 2026-08-07 06:12:29 on dist-test-slave-h3ft"
server_key: "b209cc36348b63fe71a4b77a3f82f8ec"
server_key_iv: "562c8b2f91d774c759f701104240c2e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:29.442510 20834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "9e13573b085646f9a34a548444aae625"
format_stamp: "Formatted at 2026-08-07 06:12:29 on dist-test-slave-h3ft"
server_key: "b209cc36348b63fe71a4b77a3f82f8ec"
server_key_iv: "562c8b2f91d774c759f701104240c2e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:29.447077 20834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 06:12:29.449906 20854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:29.451406 20834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:29.451553 20834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "9e13573b085646f9a34a548444aae625"
format_stamp: "Formatted at 2026-08-07 06:12:29 on dist-test-slave-h3ft"
server_key: "b209cc36348b63fe71a4b77a3f82f8ec"
server_key_iv: "562c8b2f91d774c759f701104240c2e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:29.451678 20834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:29.463241 20834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:29.466806 20834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:29.467109 20834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:29.468928 20834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:29.469015 20834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:29.469101 20834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:29.469147 20834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083149, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:29.482292 20834 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:35683
I20260807 06:12:29.482378 21017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:35683 every 8 connection(s)
I20260807 06:12:29.483534 20834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:29.486563 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.470194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:46845 (local address 127.9.133.190:41417)
0807 06:12:29.470397 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.470403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.471743 (+ 1340us) server_negotiation.cc:407] Connection header received
0807 06:12:29.473610 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.473616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.473699 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.473832 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:29.475595 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.476212 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.477583 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.478010 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.478162 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.482090 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:29.482094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:29.482096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:29.482130 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:29.484117 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.484810 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.484814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.484815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.484915 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.485206 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.485209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.485210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.485419 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:29.485506 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.486281 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.486407 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 06:12:29.487731 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.470793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.9.133.131:46845)
0807 06:12:29.471526 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.471598 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.473281 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.474101 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.474113 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.474654 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:29.475361 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.475380 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.476360 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.476363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.477423 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.477442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.478223 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.478230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.478479 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.479843 (+ 1364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:29.479862 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:29.481820 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:29.484265 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.484271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.484290 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.484706 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.484992 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.484995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.484996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.485130 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.485572 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:29.485579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:29.485966 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.486224 (+ 258us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.486600 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":217,"threads_started":1}
I20260807 06:12:29.489106 21018 heartbeater.cc:347] Connected to a master server at 127.9.133.190:41417
I20260807 06:12:29.489392 21018 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:29.490008 21018 heartbeater.cc:511] Master 127.9.133.190:41417 requested a full tablet report, sending...
I20260807 06:12:29.490546 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 20834
I20260807 06:12:29.490716 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:29.491195 9750 external_mini_cluster.cc:1428] Setting key 9823e61c1ea149d45b8e9d5015a8d2c6
I20260807 06:12:29.491393 20323 ts_manager.cc:194] Registered new tserver with Master: 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683)
I20260807 06:12:29.492271 20323 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:46845
I20260807 06:12:29.494014 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:12:29.501125 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.492589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:39005 (local address 127.9.133.190:41417)
0807 06:12:29.492757 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.492762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.493112 (+ 350us) server_negotiation.cc:407] Connection header received
0807 06:12:29.493298 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.493304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.493372 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.493461 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:29.494179 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.494739 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.495708 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.495959 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.496082 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.497432 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:29.497437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:29.497440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:29.497472 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:29.499233 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.499714 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.499720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.499722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.499818 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.500140 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.500145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.500147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.500377 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:29.500502 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.500793 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.500924 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 06:12:29.501144 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.492678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.9.133.131:39005)
0807 06:12:29.493008 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.493023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.493182 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.493495 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.493498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.493801 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:29.493974 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.493984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.494890 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.494894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.495570 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.495580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.496083 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.496087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.496189 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.496876 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:29.496891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:29.497262 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:29.499360 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.499366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.499369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.499600 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.499929 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:29.499933 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:29.499935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:29.500036 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:29.500487 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:29.500491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:29.500608 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.500732 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.500917 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":125,"threads_started":1}
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083149, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:12:29.524935 20392 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:29.517710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.9.133.190:41417)
0807 06:12:29.517909 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.517913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.517928 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:12:29.517983 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.517985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.518034 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.518158 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:29.518996 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.519632 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.520588 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.520830 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.520950 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.524701 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42630: BlockingRecv error: recv got EOF from 127.0.0.1:42630 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 06:12:29 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083149, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:29.535506 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.526597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.9.133.190:41417)
0807 06:12:29.526725 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.526728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.527075 (+ 347us) server_negotiation.cc:407] Connection header received
0807 06:12:29.527238 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.527240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.527287 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.527361 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:29.527971 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.528486 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.529281 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.529491 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.529565 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.532161 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:29.532164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:29.532166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:29.532191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:29.533769 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.534160 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.534162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.534164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.534243 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:29.534524 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:29.534528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:29.534530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:29.534761 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:29.534894 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.535241 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.535346 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:12:29.539080 20323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:29.542788 20323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:29.559999 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.553008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35683 (local address 127.0.0.1:60756)
0807 06:12:29.553966 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.553982 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.554066 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.555216 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.555223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.555248 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:29.555548 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.555559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.557087 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.557091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.558217 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.558227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.559312 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.559317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.559450 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.559522 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.559639 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.559797 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":140,"spinlock_wait_cycles":9472,"thread_start_us":123,"threads_started":1}
I20260807 06:12:29.560568 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.554031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.9.133.131:35683)
0807 06:12:29.554628 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.554642 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.554668 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:12:29.554765 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.554772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.555034 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.555225 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:29.555697 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.556938 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.558349 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.559175 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.559293 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.559491 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.560284 (+ 793us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.560397 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":167,"threads_started":1}
I20260807 06:12:29.560228 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.552991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38123 (local address 127.0.0.1:44060)
0807 06:12:29.553492 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.553553 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.553705 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.555452 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.555456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.555477 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:29.555552 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.555557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.557002 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.557007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.558581 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.558594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.559737 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.559741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.559827 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.559851 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.559898 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.559964 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":75,"thread_start_us":108,"threads_started":1}
I20260807 06:12:29.561020 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.553819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43521 (local address 127.0.0.1:44124)
0807 06:12:29.554200 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.554216 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.554311 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.555315 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.555318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.555336 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:29.555525 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.555535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.557108 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.557111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.558589 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.558594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.560570 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.560574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.560680 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.560712 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.560790 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.560862 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":207,"thread_start_us":123,"threads_started":1}
I20260807 06:12:29.560992 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.554041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.9.133.130:38123)
0807 06:12:29.554849 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.554886 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.554912 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:12:29.555043 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.555052 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.555271 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.555487 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:29.555659 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.556878 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.558728 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.559602 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.560079 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.560511 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.560570 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.560825 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":132,"threads_started":1}
I20260807 06:12:29.561446 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.554447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.9.133.129:43521)
0807 06:12:29.554768 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.554775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.554795 (+ 20us) server_negotiation.cc:407] Connection header received
0807 06:12:29.554896 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.554901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.555161 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.555679 (+ 518us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:29.555854 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.556943 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.559576 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.560420 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.560554 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.560706 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.561194 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.561307 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
I20260807 06:12:29.564527 20731 tablet_service.cc:1467] Processing CreateTablet for tablet e2ff867334d64199b52d1586917ecac4 (DEFAULT_TABLE table=test-table [id=c3e7fc1faa1f447ab1fa34756056a336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:29.564528 20927 tablet_service.cc:1467] Processing CreateTablet for tablet e2ff867334d64199b52d1586917ecac4 (DEFAULT_TABLE table=test-table [id=c3e7fc1faa1f447ab1fa34756056a336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:29.564531 20522 tablet_service.cc:1467] Processing CreateTablet for tablet e2ff867334d64199b52d1586917ecac4 (DEFAULT_TABLE table=test-table [id=c3e7fc1faa1f447ab1fa34756056a336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:29.565732 20731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ff867334d64199b52d1586917ecac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:29.565734 20522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ff867334d64199b52d1586917ecac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:29.565733 20927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ff867334d64199b52d1586917ecac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:29.574782 21059 tablet_bootstrap.cc:492] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Bootstrap starting.
I20260807 06:12:29.574782 21060 tablet_bootstrap.cc:492] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Bootstrap starting.
I20260807 06:12:29.576066 21058 tablet_bootstrap.cc:492] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Bootstrap starting.
I20260807 06:12:29.577704 21060 tablet_bootstrap.cc:654] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:29.577704 21059 tablet_bootstrap.cc:654] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:29.578020 21058 tablet_bootstrap.cc:654] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:29.578761 21058 log.cc:824] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:29.578768 21060 log.cc:824] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:29.578768 21059 log.cc:824] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:29.581169 21059 tablet_bootstrap.cc:492] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: No bootstrap required, opened a new log
I20260807 06:12:29.581176 21060 tablet_bootstrap.cc:492] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: No bootstrap required, opened a new log
I20260807 06:12:29.581403 21060 ts_tablet_manager.cc:1397] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:12:29.581406 21059 ts_tablet_manager.cc:1397] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 06:12:29.582444 21058 tablet_bootstrap.cc:492] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: No bootstrap required, opened a new log
I20260807 06:12:29.582680 21058 ts_tablet_manager.cc:1397] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 06:12:29.585500 21059 raft_consensus.cc:348] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.585815 21059 raft_consensus.cc:374] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:29.585875 21059 raft_consensus.cc:729] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Initialized, Role: FOLLOWER
I20260807 06:12:29.585752 21060 raft_consensus.cc:348] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.586051 21060 raft_consensus.cc:374] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:29.586109 21060 raft_consensus.cc:729] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e13573b085646f9a34a548444aae625, State: Initialized, Role: FOLLOWER
I20260807 06:12:29.586495 21059 consensus_queue.cc:260] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.586545 21060 consensus_queue.cc:260] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.586757 21058 raft_consensus.cc:348] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.587055 21058 raft_consensus.cc:374] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:29.587116 21058 raft_consensus.cc:729] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8897dbfe3d104807b8169aaa2a73cc7f, State: Initialized, Role: FOLLOWER
I20260807 06:12:29.587523 21058 consensus_queue.cc:260] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.587757 21059 ts_tablet_manager.cc:1428] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 06:12:29.587754 21060 ts_tablet_manager.cc:1428] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:12:29.588198 21018 heartbeater.cc:503] Master 127.9.133.190:41417 was elected leader, sending a full tablet report...
I20260807 06:12:29.588205 20627 heartbeater.cc:503] Master 127.9.133.190:41417 was elected leader, sending a full tablet report...
I20260807 06:12:29.588424 21058 ts_tablet_manager.cc:1428] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 06:12:29.588762 20824 heartbeater.cc:503] Master 127.9.133.190:41417 was elected leader, sending a full tablet report...
W20260807 06:12:29.716638 20628 tablet.cc:2369] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:29.727564 20825 tablet.cc:2369] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:29.734992 21021 tablet.cc:2369] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:12:29.874815 21068 raft_consensus.cc:482] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:29.875276 21068 raft_consensus.cc:504] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.876839 21068 leader_election.cc:288] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683), d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521)
I20260807 06:12:29.881522 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.877732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43521 (local address 127.9.133.130:40867)
0807 06:12:29.878004 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.878020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.878132 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.878407 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.878410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.878427 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:29.878569 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.878575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.879412 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.879417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.880298 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.880306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.881020 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.881023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.881127 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.881153 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.881252 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.881332 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":69,"threads_started":1}
I20260807 06:12:29.881505 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.877980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:40867 (local address 127.9.133.129:43521)
0807 06:12:29.878150 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.878158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.878176 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:12:29.878244 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.878247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.878301 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.878392 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:29.878701 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.879286 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.880439 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.880875 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.881020 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.881090 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.881270 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.881358 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 06:12:29.881755 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.877418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:40405 (local address 127.9.133.131:35683)
0807 06:12:29.877607 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.877612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.877744 (+ 132us) server_negotiation.cc:407] Connection header received
0807 06:12:29.877979 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.877982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.878043 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.878112 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:29.878548 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.879346 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.880562 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.881037 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.881197 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.881258 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.881483 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.881573 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 06:12:29.882247 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.877218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35683 (local address 127.9.133.130:40405)
0807 06:12:29.877613 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:29.877632 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:29.877817 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:29.878264 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:29.878266 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:29.878283 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:29.878409 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.878413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.879517 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.879522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.880399 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:29.880410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.881196 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.881200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:29.881329 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.881349 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 06:12:29.881451 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 06:12:29.881532 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260807 06:12:29.882332 20551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ff867334d64199b52d1586917ecac4" candidate_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e7776bf5684c1cbad259685e719276" is_pre_election: true
I20260807 06:12:29.882793 20551 raft_consensus.cc:2454] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8897dbfe3d104807b8169aaa2a73cc7f in term 0.
I20260807 06:12:29.883026 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ff867334d64199b52d1586917ecac4" candidate_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625" is_pre_election: true
I20260807 06:12:29.883370 20958 raft_consensus.cc:2454] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8897dbfe3d104807b8169aaa2a73cc7f in term 0.
I20260807 06:12:29.883509 20659 leader_election.cc:302] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8897dbfe3d104807b8169aaa2a73cc7f, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:29.883878 21068 raft_consensus.cc:2781] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:29.883970 21068 raft_consensus.cc:482] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:29.884003 21068 raft_consensus.cc:3037] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:29.885280 21068 raft_consensus.cc:504] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.885680 21068 leader_election.cc:288] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [CANDIDATE]: Term 1 election: Requested vote from peers 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683), d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521)
I20260807 06:12:29.886174 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ff867334d64199b52d1586917ecac4" candidate_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625"
I20260807 06:12:29.886300 20551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ff867334d64199b52d1586917ecac4" candidate_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e7776bf5684c1cbad259685e719276"
I20260807 06:12:29.886368 20958 raft_consensus.cc:3037] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:29.886534 20551 raft_consensus.cc:3037] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:29.888041 20551 raft_consensus.cc:2454] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8897dbfe3d104807b8169aaa2a73cc7f in term 1.
I20260807 06:12:29.888227 20958 raft_consensus.cc:2454] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8897dbfe3d104807b8169aaa2a73cc7f in term 1.
I20260807 06:12:29.888461 20659 leader_election.cc:302] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8897dbfe3d104807b8169aaa2a73cc7f, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:29.888695 21068 raft_consensus.cc:2781] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:29.888962 21068 raft_consensus.cc:686] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 LEADER]: Becoming Leader. State: Replica: 8897dbfe3d104807b8169aaa2a73cc7f, State: Running, Role: LEADER
I20260807 06:12:29.889428 21068 consensus_queue.cc:237] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } }
I20260807 06:12:29.894217 20322 catalog_manager.cc:5495] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130). New cstate: current_term: 1 leader_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } health_report { overall_health: UNKNOWN } } }
I20260807 06:12:29.928292 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:29.924143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.9.133.130:38123)
0807 06:12:29.924303 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:29.924307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:29.924322 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:12:29.924441 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:29.924444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:29.924522 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:29.924619 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:29.924886 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.925480 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.926550 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:29.926818 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:29.926907 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:29.927226 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:29.927836 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:29.927935 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:29.928020 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 06:12:29.928079 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:12:29.936174 20551 raft_consensus.cc:1261] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Refusing update from remote peer 8897dbfe3d104807b8169aaa2a73cc7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:29.936174 20958 raft_consensus.cc:1261] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Refusing update from remote peer 8897dbfe3d104807b8169aaa2a73cc7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:29.937072 21068 consensus_queue.cc:1035] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:29.937325 21075 consensus_queue.cc:1035] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:29.947443 21083 mvcc.cc:205] Tried to move back new op lower bound from 7315796582128615424 to 7315796581950390272. Current Snapshot: MvccSnapshot[applied={T|T < 7315796582128615424}]
I20260807 06:12:29.947890 21085 mvcc.cc:205] Tried to move back new op lower bound from 7315796582128615424 to 7315796581950390272. Current Snapshot: MvccSnapshot[applied={T|T < 7315796582128615424}]
I20260807 06:12:29.958038 20323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:29.958315 20323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:29.961591 20323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Sending DeleteTablet for 3 replicas of tablet e2ff867334d64199b52d1586917ecac4
I20260807 06:12:29.962646 20731 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ff867334d64199b52d1586917ecac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:29 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44060
I20260807 06:12:29.962932 20522 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ff867334d64199b52d1586917ecac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:29 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44124
I20260807 06:12:29.963474 20927 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ff867334d64199b52d1586917ecac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:29 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:60756
I20260807 06:12:29.964000 21103 tablet_replica.cc:330] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: stopping tablet replica
I20260807 06:12:29.964488 21103 raft_consensus.cc:2229] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:29.964742 20323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 06:12:29.965304 21103 raft_consensus.cc:2258] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 06:12:29.965343 20323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:29.967301 21105 tablet_replica.cc:330] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: stopping tablet replica
I20260807 06:12:29.967720 21105 raft_consensus.cc:2229] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:29.968125 21105 raft_consensus.cc:2258] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:29.968568 21104 tablet_replica.cc:330] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: stopping tablet replica
I20260807 06:12:29.968864 21104 raft_consensus.cc:2229] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:29.969157 21104 raft_consensus.cc:2258] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:29.971119 21105 ts_tablet_manager.cc:1905] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:29.973258 20522 tablet_service.cc:1467] Processing CreateTablet for tablet a0927dc6770a4fe5b8509998fdc9742d (DEFAULT_TABLE table=test-table [id=e129d0d4ad4e4858a1fc4ae5ad3a46ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:29.973318 20927 tablet_service.cc:1467] Processing CreateTablet for tablet a0927dc6770a4fe5b8509998fdc9742d (DEFAULT_TABLE table=test-table [id=e129d0d4ad4e4858a1fc4ae5ad3a46ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:29.973384 20731 tablet_service.cc:1467] Processing CreateTablet for tablet a0927dc6770a4fe5b8509998fdc9742d (DEFAULT_TABLE table=test-table [id=e129d0d4ad4e4858a1fc4ae5ad3a46ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:29.973620 20522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0927dc6770a4fe5b8509998fdc9742d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:29.973661 20927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0927dc6770a4fe5b8509998fdc9742d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:29.973666 20731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0927dc6770a4fe5b8509998fdc9742d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:29.976552 21104 ts_tablet_manager.cc:1905] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:29.976706 21058 tablet_bootstrap.cc:492] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Bootstrap starting.
I20260807 06:12:29.977871 21058 tablet_bootstrap.cc:654] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:29.978611 21103 ts_tablet_manager.cc:1905] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:29.978811 21059 tablet_bootstrap.cc:492] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Bootstrap starting.
I20260807 06:12:29.980064 21059 tablet_bootstrap.cc:654] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:29.981323 21060 tablet_bootstrap.cc:492] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Bootstrap starting.
I20260807 06:12:29.982481 21060 tablet_bootstrap.cc:654] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:29.985160 21105 ts_tablet_manager.cc:1918] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:29.985280 21105 log.cc:1196] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/e2ff867334d64199b52d1586917ecac4
I20260807 06:12:29.985679 21105 ts_tablet_manager.cc:1939] T e2ff867334d64199b52d1586917ecac4 P 9e13573b085646f9a34a548444aae625: Deleting consensus metadata
I20260807 06:12:29.986661 21103 ts_tablet_manager.cc:1918] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:29.986860 21103 log.cc:1196] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/e2ff867334d64199b52d1586917ecac4
I20260807 06:12:29.987349 21103 ts_tablet_manager.cc:1939] T e2ff867334d64199b52d1586917ecac4 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting consensus metadata
I20260807 06:12:29.987388 20297 catalog_manager.cc:4841] TS 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683): tablet e2ff867334d64199b52d1586917ecac4 (table test-table [id=c3e7fc1faa1f447ab1fa34756056a336]) successfully deleted
I20260807 06:12:29.988490 21104 ts_tablet_manager.cc:1918] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:29.988618 21104 log.cc:1196] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/e2ff867334d64199b52d1586917ecac4
I20260807 06:12:29.988982 21104 ts_tablet_manager.cc:1939] T e2ff867334d64199b52d1586917ecac4 P d5e7776bf5684c1cbad259685e719276: Deleting consensus metadata
I20260807 06:12:29.990027 20299 catalog_manager.cc:4841] TS 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123): tablet e2ff867334d64199b52d1586917ecac4 (table test-table [id=c3e7fc1faa1f447ab1fa34756056a336]) successfully deleted
I20260807 06:12:29.990410 20297 catalog_manager.cc:4841] TS d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521): tablet e2ff867334d64199b52d1586917ecac4 (table test-table [id=c3e7fc1faa1f447ab1fa34756056a336]) successfully deleted
I20260807 06:12:29.990492 21058 tablet_bootstrap.cc:492] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: No bootstrap required, opened a new log
I20260807 06:12:29.990602 21058 ts_tablet_manager.cc:1397] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20260807 06:12:29.991303 21059 tablet_bootstrap.cc:492] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: No bootstrap required, opened a new log
I20260807 06:12:29.991453 21059 ts_tablet_manager.cc:1397] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260807 06:12:29.991571 21060 tablet_bootstrap.cc:492] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: No bootstrap required, opened a new log
I20260807 06:12:29.991674 21060 ts_tablet_manager.cc:1397] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260807 06:12:29.991818 21058 raft_consensus.cc:348] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:29.991981 21059 raft_consensus.cc:348] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:29.992094 21059 raft_consensus.cc:374] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:29.992096 21058 raft_consensus.cc:374] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:29.992136 21059 raft_consensus.cc:729] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Initialized, Role: FOLLOWER
I20260807 06:12:29.992139 21058 raft_consensus.cc:729] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8897dbfe3d104807b8169aaa2a73cc7f, State: Initialized, Role: FOLLOWER
I20260807 06:12:29.992220 21060 raft_consensus.cc:348] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:29.992280 21059 consensus_queue.cc:260] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:29.992280 21058 consensus_queue.cc:260] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:29.992365 21060 raft_consensus.cc:374] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:29.992405 21060 raft_consensus.cc:729] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e13573b085646f9a34a548444aae625, State: Initialized, Role: FOLLOWER
I20260807 06:12:29.992547 21060 consensus_queue.cc:260] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:29.992631 21059 ts_tablet_manager.cc:1428] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 06:12:29.992630 21058 ts_tablet_manager.cc:1428] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 06:12:29.992861 21060 ts_tablet_manager.cc:1428] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:30.046804 21066 raft_consensus.cc:482] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:30.047065 21066 raft_consensus.cc:504] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.048759 21066 leader_election.cc:288] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521), 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123)
I20260807 06:12:30.051514 21065 raft_consensus.cc:482] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:30.051702 21065 raft_consensus.cc:504] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.053279 21065 leader_election.cc:288] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123), 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683)
I20260807 06:12:30.054610 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.049234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:56503 (local address 127.9.133.130:38123)
0807 06:12:30.049480 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.049487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.049764 (+ 277us) server_negotiation.cc:407] Connection header received
0807 06:12:30.050309 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.050313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.050369 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.050459 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:30.050802 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.051490 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.053108 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.053722 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.053839 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.053912 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.054321 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.054429 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 06:12:30.054785 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.049318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:60369 (local address 127.9.133.129:43521)
0807 06:12:30.049460 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.049467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.049819 (+ 352us) server_negotiation.cc:407] Connection header received
0807 06:12:30.049958 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.049961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.050028 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.050128 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:30.050614 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.051524 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.053315 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.053946 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.054042 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.054093 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.054559 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.054653 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:12:30.055126 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.049043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38123 (local address 127.9.133.131:56503)
0807 06:12:30.049655 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.049669 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.049823 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.050508 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.050513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.050535 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:30.050677 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.050683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.051656 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.051660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.052941 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.052954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.054038 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.054041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.054158 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.054185 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.054783 (+ 598us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.054889 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":242,"spinlock_wait_cycles":31232,"thread_start_us":74,"threads_started":1}
I20260807 06:12:30.056057 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0927dc6770a4fe5b8509998fdc9742d" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" is_pre_election: true
I20260807 06:12:30.056391 20761 raft_consensus.cc:2454] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 0.
I20260807 06:12:30.056929 20858 leader_election.cc:302] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8897dbfe3d104807b8169aaa2a73cc7f, 9e13573b085646f9a34a548444aae625; no voters:
I20260807 06:12:30.057271 21066 raft_consensus.cc:2781] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:30.057336 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.049043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43521 (local address 127.9.133.131:60369)
0807 06:12:30.049639 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.049655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.049823 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.050218 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.050223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.050250 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:30.050408 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.050414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.051785 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.051789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.053154 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.053168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.054284 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.054288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.054400 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.054436 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.056197 (+ 1761us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.056384 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"spinlock_wait_cycles":23808,"thread_start_us":120,"threads_started":1}
I20260807 06:12:30.057340 21066 raft_consensus.cc:482] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:30.057614 21066 raft_consensus.cc:3037] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.057823 20551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0927dc6770a4fe5b8509998fdc9742d" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e7776bf5684c1cbad259685e719276" is_pre_election: true
I20260807 06:12:30.057982 20551 raft_consensus.cc:2454] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 0.
I20260807 06:12:30.059393 21066 raft_consensus.cc:504] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.059854 21066 leader_election.cc:288] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 election: Requested vote from peers d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521), 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123)
I20260807 06:12:30.060148 20551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0927dc6770a4fe5b8509998fdc9742d" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e7776bf5684c1cbad259685e719276"
I20260807 06:12:30.060287 20551 raft_consensus.cc:3037] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.061412 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.054079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:35189 (local address 127.9.133.131:35683)
0807 06:12:30.054211 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.054217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.055087 (+ 870us) server_negotiation.cc:407] Connection header received
0807 06:12:30.055706 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.055710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.055761 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.055813 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:30.056227 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.056907 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.060096 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.060686 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.060782 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.060835 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.061210 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.061292 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:12:30.061828 20551 raft_consensus.cc:2454] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 1.
I20260807 06:12:30.061954 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.055191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:57347 (local address 127.9.133.130:38123)
0807 06:12:30.055935 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.055939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.055953 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:30.055997 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.056000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.056048 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.056120 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:30.057664 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.058394 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.060508 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.061100 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.061188 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.061241 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.061748 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.061829 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661}
I20260807 06:12:30.061976 21113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.053693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35683 (local address 127.9.133.129:35189)
0807 06:12:30.054964 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.054981 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.055594 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.055948 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.055951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.055970 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:30.056095 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.056103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.057110 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.057116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.059666 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.059676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.060950 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.060954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.061064 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.061088 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.061618 (+ 530us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.061725 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"mutex_wait_us":38,"thread_start_us":95,"threads_started":1}
I20260807 06:12:30.062498 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.054449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38123 (local address 127.9.133.129:57347)
0807 06:12:30.055338 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.055349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.055447 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.056577 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.056580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.056598 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:30.057531 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.057542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.058540 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.058543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.060356 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.060367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.061348 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.061352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.061462 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.061618 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.062253 (+ 635us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.062336 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":174,"thread_start_us":155,"threads_started":1}
I20260807 06:12:30.062846 20856 leader_election.cc:302] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e13573b085646f9a34a548444aae625, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.062937 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0927dc6770a4fe5b8509998fdc9742d" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
I20260807 06:12:30.063062 21066 raft_consensus.cc:2781] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:30.063112 20761 raft_consensus.cc:3037] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.063256 21066 raft_consensus.cc:686] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 1 LEADER]: Becoming Leader. State: Replica: 9e13573b085646f9a34a548444aae625, State: Running, Role: LEADER
I20260807 06:12:30.063535 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0927dc6770a4fe5b8509998fdc9742d" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625" is_pre_election: true
I20260807 06:12:30.063596 21066 consensus_queue.cc:237] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.064421 20760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0927dc6770a4fe5b8509998fdc9742d" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" is_pre_election: true
I20260807 06:12:30.064532 20761 raft_consensus.cc:2454] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 1.
I20260807 06:12:30.065376 20451 leader_election.cc:302] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5e7776bf5684c1cbad259685e719276; no voters: 8897dbfe3d104807b8169aaa2a73cc7f, 9e13573b085646f9a34a548444aae625
I20260807 06:12:30.065781 21065 raft_consensus.cc:2726] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 06:12:30.067209 20320 catalog_manager.cc:5495] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e13573b085646f9a34a548444aae625 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } health_report { overall_health: HEALTHY } } }
Aug 07 06:12:30 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083148, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.190@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:30.124481 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.113658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:41417 (local address 127.0.0.1:42660)
0807 06:12:30.114063 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.114082 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.114238 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.114606 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.114608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.115212 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:30.115338 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.115345 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.116550 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.116553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.117132 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.117140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.117756 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.117758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.117822 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.118598 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:30.118612 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:30.120452 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:30.122601 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:30.122611 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:30.122617 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:30.122881 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:30.123224 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:30.123227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:30.123228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:30.123338 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:30.123854 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:30.123861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:30.123994 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.124100 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.124253 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":192,"threads_started":1}
I20260807 06:12:30.124524 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.113764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.9.133.190:41417)
0807 06:12:30.114101 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.114111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.114206 (+ 95us) server_negotiation.cc:407] Connection header received
0807 06:12:30.114357 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.114361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.114424 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.114560 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:30.115510 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.116409 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.117268 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.117622 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.117734 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.120613 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:30.120617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:30.120620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:30.120646 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:30.122445 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:30.122988 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:30.122992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:30.122994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:30.123119 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:30.123487 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:30.123490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:30.123491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:30.123729 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:30.123827 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.124180 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.124330 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260807 06:12:30.130110 20320 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:42660:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 06:12:30.131285 20320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:30.137775 20522 tablet_service.cc:1467] Processing CreateTablet for tablet d51c2df0474342219a801358060ec154 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6165238adb314cd8b98706d6c2d406dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:12:30.137773 20731 tablet_service.cc:1467] Processing CreateTablet for tablet d51c2df0474342219a801358060ec154 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6165238adb314cd8b98706d6c2d406dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:12:30.138118 20927 tablet_service.cc:1467] Processing CreateTablet for tablet d51c2df0474342219a801358060ec154 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6165238adb314cd8b98706d6c2d406dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 06:12:30.138360 20731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51c2df0474342219a801358060ec154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.138368 20522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51c2df0474342219a801358060ec154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.138551 20927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51c2df0474342219a801358060ec154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.141913 21058 tablet_bootstrap.cc:492] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: Bootstrap starting.
I20260807 06:12:30.142010 21059 tablet_bootstrap.cc:492] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: Bootstrap starting.
I20260807 06:12:30.143031 21058 tablet_bootstrap.cc:654] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.143034 21059 tablet_bootstrap.cc:654] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.143930 21060 tablet_bootstrap.cc:492] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: Bootstrap starting.
I20260807 06:12:30.145167 21060 tablet_bootstrap.cc:654] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.145283 21059 tablet_bootstrap.cc:492] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: No bootstrap required, opened a new log
I20260807 06:12:30.145390 21059 ts_tablet_manager.cc:1397] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:30.145466 21058 tablet_bootstrap.cc:492] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: No bootstrap required, opened a new log
I20260807 06:12:30.145581 21058 ts_tablet_manager.cc:1397] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:12:30.145991 21058 raft_consensus.cc:348] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.146005 21059 raft_consensus.cc:348] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.146143 21059 raft_consensus.cc:374] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.146144 21058 raft_consensus.cc:374] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.146181 21059 raft_consensus.cc:729] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.146181 21058 raft_consensus.cc:729] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8897dbfe3d104807b8169aaa2a73cc7f, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.146327 21059 consensus_queue.cc:260] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.146327 21058 consensus_queue.cc:260] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.146642 21058 ts_tablet_manager.cc:1428] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:30.146642 21065 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.146843 21065 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.146924 21068 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.147106 21068 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.147923 21060 tablet_bootstrap.cc:492] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: No bootstrap required, opened a new log
I20260807 06:12:30.148025 21060 ts_tablet_manager.cc:1397] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 06:12:30.148463 21059 ts_tablet_manager.cc:1428] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 06:12:30.148597 21060 raft_consensus.cc:348] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.148722 21060 raft_consensus.cc:374] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.148752 21060 raft_consensus.cc:729] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e13573b085646f9a34a548444aae625, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.148890 21060 consensus_queue.cc:260] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.149158 21060 ts_tablet_manager.cc:1428] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:30.149463 21066 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.149621 21066 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.397401 21066 raft_consensus.cc:482] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:30.397608 21066 raft_consensus.cc:504] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.398295 21066 leader_election.cc:288] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521), 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123)
I20260807 06:12:30.398907 20760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51c2df0474342219a801358060ec154" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" is_pre_election: true
I20260807 06:12:30.398908 20551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51c2df0474342219a801358060ec154" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e7776bf5684c1cbad259685e719276" is_pre_election: true
I20260807 06:12:30.399159 20551 raft_consensus.cc:2454] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 0.
I20260807 06:12:30.399159 20760 raft_consensus.cc:2454] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 0.
I20260807 06:12:30.399596 20858 leader_election.cc:302] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8897dbfe3d104807b8169aaa2a73cc7f, 9e13573b085646f9a34a548444aae625, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.399813 21066 raft_consensus.cc:2781] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:30.399885 21066 raft_consensus.cc:482] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:30.399914 21066 raft_consensus.cc:3037] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.401232 21066 raft_consensus.cc:504] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.401618 21066 leader_election.cc:288] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 election: Requested vote from peers d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521), 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123)
I20260807 06:12:30.402042 20551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51c2df0474342219a801358060ec154" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5e7776bf5684c1cbad259685e719276"
I20260807 06:12:30.402082 20760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51c2df0474342219a801358060ec154" candidate_uuid: "9e13573b085646f9a34a548444aae625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
I20260807 06:12:30.402218 20760 raft_consensus.cc:3037] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.402220 20551 raft_consensus.cc:3037] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.403612 20551 raft_consensus.cc:2454] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 1.
I20260807 06:12:30.403753 20760 raft_consensus.cc:2454] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e13573b085646f9a34a548444aae625 in term 1.
I20260807 06:12:30.404006 20856 leader_election.cc:302] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e13573b085646f9a34a548444aae625, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.404181 21066 raft_consensus.cc:2781] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:30.404310 21066 raft_consensus.cc:686] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [term 1 LEADER]: Becoming Leader. State: Replica: 9e13573b085646f9a34a548444aae625, State: Running, Role: LEADER
I20260807 06:12:30.404484 21066 consensus_queue.cc:237] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.405580 21115 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: TxnStatusTablet state changed. Reason: New leader 9e13573b085646f9a34a548444aae625. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.405712 21115 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:12:30.406111 21131 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 06:12:30.406315 21131 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 06:12:30.406813 20320 catalog_manager.cc:5495] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e13573b085646f9a34a548444aae625 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } health_report { overall_health: HEALTHY } } }
I20260807 06:12:30.456528 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.451616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60766 (local address 127.9.133.131:35683)
0807 06:12:30.451766 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.451770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.451869 (+ 99us) server_negotiation.cc:407] Connection header received
0807 06:12:30.452100 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.452103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.452172 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.452274 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:30.452616 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.453177 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.454492 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.454718 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.454841 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.455325 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.456016 (+ 691us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.456170 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.456237 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.456303 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:12:30.456519 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.451471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35683 (local address 127.0.0.1:60766)
0807 06:12:30.451787 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.451804 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.451936 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.452311 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.452314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.452323 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:12:30.452485 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.452494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.453293 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.453299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.454341 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.454353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.454851 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.454855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.454968 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.455068 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.456122 (+ 1054us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.456145 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.456223 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.456363 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260807 06:12:30.461035 20551 raft_consensus.cc:1261] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Refusing update from remote peer 9e13573b085646f9a34a548444aae625: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.461185 20760 raft_consensus.cc:1261] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Refusing update from remote peer 9e13573b085646f9a34a548444aae625: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.461717 21115 consensus_queue.cc:1035] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.462224 21066 consensus_queue.cc:1035] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.463872 21065 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: TxnStatusTablet state changed. Reason: New leader 9e13573b085646f9a34a548444aae625. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.464027 21065 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.464416 21068 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: TxnStatusTablet state changed. Reason: New leader 9e13573b085646f9a34a548444aae625. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.464563 21068 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.467654 21115 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.467772 21115 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:12:30.467900 21115 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.467975 21115 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 9e13573b085646f9a34a548444aae625: This TxnStatusTablet replica's current role is: LEADER
I20260807 06:12:30.469422 21065 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.469554 21065 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P d5e7776bf5684c1cbad259685e719276: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.470291 21135 mvcc.cc:205] Tried to move back new op lower bound from 7315796584281436160 to 7315796584058474496. Current Snapshot: MvccSnapshot[applied={T|T < 7315796584281436160 or (T in {7315796584281436160})}]
I20260807 06:12:30.471095 21068 tablet_replica.cc:439] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e13573b085646f9a34a548444aae625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } } }
I20260807 06:12:30.471235 21068 tablet_replica.cc:442] T d51c2df0474342219a801358060ec154 P 8897dbfe3d104807b8169aaa2a73cc7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 06:12:30.477259 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.473369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.9.133.131:35683)
0807 06:12:30.473504 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.473509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.473837 (+ 328us) server_negotiation.cc:407] Connection header received
0807 06:12:30.474021 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.474025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.474072 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.474157 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:30.474436 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.475119 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.476053 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.476291 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.476403 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.476701 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.476783 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.476865 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.477024 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.477080 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:12:30.490118 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.485816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35683 (local address 127.9.133.131:51175)
0807 06:12:30.486179 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.486199 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.486319 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.486690 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.486693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.486719 (+ 26us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:12:30.486882 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.486890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.487701 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.487708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.488481 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.488492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.489242 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.489254 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.489322 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.489377 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.489702 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.489710 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.489784 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.489936 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":55,"thread_start_us":143,"threads_started":1}
I20260807 06:12:30.490118 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.486179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:51175 (local address 127.9.133.131:35683)
0807 06:12:30.486291 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.486296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.486310 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:30.486428 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.486432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.486486 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.486577 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:30.487062 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.487581 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.488617 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.489045 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.489154 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.489521 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.489610 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.489700 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.489792 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.489882 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:12:30.498363 20551 raft_consensus.cc:1261] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Refusing update from remote peer 9e13573b085646f9a34a548444aae625: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.498558 20760 raft_consensus.cc:1261] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Refusing update from remote peer 9e13573b085646f9a34a548444aae625: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.498735 21066 consensus_queue.cc:1035] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.499197 21066 consensus_queue.cc:1035] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.542924 20323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:30.543219 20323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:30.545773 20323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Sending DeleteTablet for 3 replicas of tablet a0927dc6770a4fe5b8509998fdc9742d
I20260807 06:12:30.546558 20522 tablet_service.cc:1514] Processing DeleteTablet for tablet a0927dc6770a4fe5b8509998fdc9742d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44124
I20260807 06:12:30.546598 20925 tablet_service.cc:1514] Processing DeleteTablet for tablet a0927dc6770a4fe5b8509998fdc9742d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:60756
I20260807 06:12:30.547022 21154 tablet_replica.cc:330] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: stopping tablet replica
I20260807 06:12:30.546980 21155 tablet_replica.cc:330] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: stopping tablet replica
I20260807 06:12:30.547163 21155 raft_consensus.cc:2229] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:30.547163 21154 raft_consensus.cc:2229] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:30.547293 21154 raft_consensus.cc:2258] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.547616 21155 raft_consensus.cc:2258] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.548174 21154 ts_tablet_manager.cc:1905] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.548282 21155 ts_tablet_manager.cc:1905] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.551597 21154 ts_tablet_manager.cc:1918] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:12:30.551597 21155 ts_tablet_manager.cc:1918] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:12:30.551680 21155 log.cc:1196] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/a0927dc6770a4fe5b8509998fdc9742d
I20260807 06:12:30.551689 21154 log.cc:1196] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/a0927dc6770a4fe5b8509998fdc9742d
I20260807 06:12:30.551994 21155 ts_tablet_manager.cc:1939] T a0927dc6770a4fe5b8509998fdc9742d P 9e13573b085646f9a34a548444aae625: Deleting consensus metadata
I20260807 06:12:30.552383 21154 ts_tablet_manager.cc:1939] T a0927dc6770a4fe5b8509998fdc9742d P d5e7776bf5684c1cbad259685e719276: Deleting consensus metadata
I20260807 06:12:30.552877 20297 catalog_manager.cc:4841] TS 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683): tablet a0927dc6770a4fe5b8509998fdc9742d (table test-table [id=e129d0d4ad4e4858a1fc4ae5ad3a46ad]) successfully deleted
I20260807 06:12:30.553319 20731 tablet_service.cc:1514] Processing DeleteTablet for tablet a0927dc6770a4fe5b8509998fdc9742d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44060
I20260807 06:12:30.553491 20297 catalog_manager.cc:4841] TS d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521): tablet a0927dc6770a4fe5b8509998fdc9742d (table test-table [id=e129d0d4ad4e4858a1fc4ae5ad3a46ad]) successfully deleted
I20260807 06:12:30.553821 21158 tablet_replica.cc:330] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: stopping tablet replica
I20260807 06:12:30.554004 21158 raft_consensus.cc:2229] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:30.554172 21158 raft_consensus.cc:2258] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.554991 21158 ts_tablet_manager.cc:1905] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.558859 21158 ts_tablet_manager.cc:1918] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:12:30.558972 21158 log.cc:1196] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/a0927dc6770a4fe5b8509998fdc9742d
I20260807 06:12:30.559902 21158 ts_tablet_manager.cc:1939] T a0927dc6770a4fe5b8509998fdc9742d P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting consensus metadata
I20260807 06:12:30.560786 20299 catalog_manager.cc:4841] TS 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123): tablet a0927dc6770a4fe5b8509998fdc9742d (table test-table [id=e129d0d4ad4e4858a1fc4ae5ad3a46ad]) successfully deleted
Aug 07 06:12:30 dist-test-slave-h3ft krb5kdc[20258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 06:12:30 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:30 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:12:30.588233 21126 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:30.579981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.9.133.190:41417)
0807 06:12:30.580182 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.580188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.580339 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:12:30.580546 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.580550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.580608 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.580678 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:30.581742 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.582339 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.583687 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.583941 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.584038 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.588062 (+ 4024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42664: BlockingRecv error: recv got EOF from 127.0.0.1:42664 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 06:12:30 dist-test-slave-h3ft krb5kdc[20258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM
I20260807 06:12:30.600502 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.590433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.9.133.190:41417)
0807 06:12:30.590568 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.590574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.590707 (+ 133us) server_negotiation.cc:407] Connection header received
0807 06:12:30.590867 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.590869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.590915 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.590992 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:30.591915 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.592465 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.593673 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.594021 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.594107 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.597033 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:30.597037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:30.597039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:30.597064 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:30.598659 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:30.599138 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:30.599142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:30.599144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:30.599233 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:30.599534 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:30.599537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:30.599541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:30.599738 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:30.599894 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.600157 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.600306 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:12:30.603585 20323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:30.604135 20323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:30.610301 20522 tablet_service.cc:1467] Processing CreateTablet for tablet 06225b0fa3414cbbbbf2b907fac36d74 (DEFAULT_TABLE table=test-table [id=fc5ff76055c54312abd4b10cf1cdabf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:30.610507 20731 tablet_service.cc:1467] Processing CreateTablet for tablet 06225b0fa3414cbbbbf2b907fac36d74 (DEFAULT_TABLE table=test-table [id=fc5ff76055c54312abd4b10cf1cdabf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:30.610744 20522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06225b0fa3414cbbbbf2b907fac36d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.610845 20731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06225b0fa3414cbbbbf2b907fac36d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.613727 21059 tablet_bootstrap.cc:492] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Bootstrap starting.
I20260807 06:12:30.614624 21059 tablet_bootstrap.cc:654] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.615942 20925 tablet_service.cc:1467] Processing CreateTablet for tablet 06225b0fa3414cbbbbf2b907fac36d74 (DEFAULT_TABLE table=test-table [id=fc5ff76055c54312abd4b10cf1cdabf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:30.616241 20925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06225b0fa3414cbbbbf2b907fac36d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.618412 21060 tablet_bootstrap.cc:492] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Bootstrap starting.
I20260807 06:12:30.619396 21059 tablet_bootstrap.cc:492] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: No bootstrap required, opened a new log
I20260807 06:12:30.619438 21060 tablet_bootstrap.cc:654] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.619515 21059 ts_tablet_manager.cc:1397] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 06:12:30.620116 21058 tablet_bootstrap.cc:492] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Bootstrap starting.
I20260807 06:12:30.620113 21059 raft_consensus.cc:348] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.620257 21059 raft_consensus.cc:374] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.620297 21059 raft_consensus.cc:729] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.620474 21059 consensus_queue.cc:260] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.621111 21058 tablet_bootstrap.cc:654] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.622941 21059 ts_tablet_manager.cc:1428] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:30.623067 21060 tablet_bootstrap.cc:492] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: No bootstrap required, opened a new log
I20260807 06:12:30.623169 21060 ts_tablet_manager.cc:1397] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 06:12:30.623641 21060 raft_consensus.cc:348] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.623778 21060 raft_consensus.cc:374] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.623821 21060 raft_consensus.cc:729] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e13573b085646f9a34a548444aae625, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.623965 21060 consensus_queue.cc:260] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.624269 21060 ts_tablet_manager.cc:1428] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 06:12:30.625476 21058 tablet_bootstrap.cc:492] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: No bootstrap required, opened a new log
I20260807 06:12:30.625589 21058 ts_tablet_manager.cc:1397] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 06:12:30.626070 21058 raft_consensus.cc:348] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.626212 21058 raft_consensus.cc:374] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.626267 21058 raft_consensus.cc:729] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8897dbfe3d104807b8169aaa2a73cc7f, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.626420 21058 consensus_queue.cc:260] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.626777 21058 ts_tablet_manager.cc:1428] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 06:12:30.675436 21065 raft_consensus.cc:482] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:30.675633 21065 raft_consensus.cc:504] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.676221 21065 leader_election.cc:288] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123), 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683)
I20260807 06:12:30.676724 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06225b0fa3414cbbbbf2b907fac36d74" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" is_pre_election: true
I20260807 06:12:30.676783 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06225b0fa3414cbbbbf2b907fac36d74" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625" is_pre_election: true
I20260807 06:12:30.676934 20761 raft_consensus.cc:2454] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 0.
I20260807 06:12:30.676947 20958 raft_consensus.cc:2454] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 0.
I20260807 06:12:30.677378 20453 leader_election.cc:302] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8897dbfe3d104807b8169aaa2a73cc7f, 9e13573b085646f9a34a548444aae625, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.677564 21065 raft_consensus.cc:2781] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:30.677631 21065 raft_consensus.cc:482] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:30.677675 21065 raft_consensus.cc:3037] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.678934 21065 raft_consensus.cc:504] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.679411 21065 leader_election.cc:288] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 election: Requested vote from peers 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123), 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683)
I20260807 06:12:30.679780 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06225b0fa3414cbbbbf2b907fac36d74" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
I20260807 06:12:30.679819 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06225b0fa3414cbbbbf2b907fac36d74" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625"
I20260807 06:12:30.679929 20761 raft_consensus.cc:3037] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.679984 20958 raft_consensus.cc:3037] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.681063 20958 raft_consensus.cc:2454] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 1.
I20260807 06:12:30.681396 20451 leader_election.cc:302] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e13573b085646f9a34a548444aae625, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.681438 20761 raft_consensus.cc:2454] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 1.
I20260807 06:12:30.681612 21065 raft_consensus.cc:2781] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:30.681862 21065 raft_consensus.cc:686] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 1 LEADER]: Becoming Leader. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Running, Role: LEADER
I20260807 06:12:30.682214 21065 consensus_queue.cc:237] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.685124 20323 catalog_manager.cc:5495] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5e7776bf5684c1cbad259685e719276 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "d5e7776bf5684c1cbad259685e719276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } health_report { overall_health: UNKNOWN } } }
I20260807 06:12:30.710520 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.705987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.9.133.129:43521)
0807 06:12:30.706327 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.706331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.706345 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:30.706453 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.706456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.706532 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.706689 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:30.707053 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.707668 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.708633 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.708898 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.709024 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.709374 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.710018 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.710168 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.710212 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.710290 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
I20260807 06:12:30.714416 20958 raft_consensus.cc:1261] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Refusing update from remote peer d5e7776bf5684c1cbad259685e719276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.714461 20761 raft_consensus.cc:1261] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Refusing update from remote peer d5e7776bf5684c1cbad259685e719276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.715090 21177 consensus_queue.cc:1035] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.715453 21065 consensus_queue.cc:1035] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.732440 20323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:30.732707 20323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:30.735736 20323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Sending DeleteTablet for 3 replicas of tablet 06225b0fa3414cbbbbf2b907fac36d74
I20260807 06:12:30.736375 20522 tablet_service.cc:1514] Processing DeleteTablet for tablet 06225b0fa3414cbbbbf2b907fac36d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44124
I20260807 06:12:30.736460 20731 tablet_service.cc:1514] Processing DeleteTablet for tablet 06225b0fa3414cbbbbf2b907fac36d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44060
I20260807 06:12:30.736619 21154 tablet_replica.cc:330] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: stopping tablet replica
I20260807 06:12:30.736689 21158 tablet_replica.cc:330] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: stopping tablet replica
I20260807 06:12:30.736799 21154 raft_consensus.cc:2229] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:30.736855 21158 raft_consensus.cc:2229] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:30.737056 21158 raft_consensus.cc:2258] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.737187 21154 raft_consensus.cc:2258] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.737954 20925 tablet_service.cc:1514] Processing DeleteTablet for tablet 06225b0fa3414cbbbbf2b907fac36d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:60756
I20260807 06:12:30.738394 21154 ts_tablet_manager.cc:1905] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.740157 21158 ts_tablet_manager.cc:1905] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.741623 21154 ts_tablet_manager.cc:1918] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:30.741712 21154 log.cc:1196] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/06225b0fa3414cbbbbf2b907fac36d74
I20260807 06:12:30.742003 21154 ts_tablet_manager.cc:1939] T 06225b0fa3414cbbbbf2b907fac36d74 P d5e7776bf5684c1cbad259685e719276: Deleting consensus metadata
I20260807 06:12:30.742895 21155 tablet_replica.cc:330] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: stopping tablet replica
I20260807 06:12:30.743093 21155 raft_consensus.cc:2229] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:30.743266 21155 raft_consensus.cc:2258] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.743544 21158 ts_tablet_manager.cc:1918] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:30.743655 21158 log.cc:1196] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/06225b0fa3414cbbbbf2b907fac36d74
I20260807 06:12:30.743819 20297 catalog_manager.cc:4841] TS d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521): tablet 06225b0fa3414cbbbbf2b907fac36d74 (table test-table [id=fc5ff76055c54312abd4b10cf1cdabf0]) successfully deleted
I20260807 06:12:30.743968 21158 ts_tablet_manager.cc:1939] T 06225b0fa3414cbbbbf2b907fac36d74 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting consensus metadata
I20260807 06:12:30.744839 21155 ts_tablet_manager.cc:1905] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.744877 20299 catalog_manager.cc:4841] TS 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123): tablet 06225b0fa3414cbbbbf2b907fac36d74 (table test-table [id=fc5ff76055c54312abd4b10cf1cdabf0]) successfully deleted
I20260807 06:12:30.745618 20320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:30.746304 20320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:30.747637 21155 ts_tablet_manager.cc:1918] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:30.747720 21155 log.cc:1196] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/06225b0fa3414cbbbbf2b907fac36d74
I20260807 06:12:30.747983 21155 ts_tablet_manager.cc:1939] T 06225b0fa3414cbbbbf2b907fac36d74 P 9e13573b085646f9a34a548444aae625: Deleting consensus metadata
I20260807 06:12:30.748769 20297 catalog_manager.cc:4841] TS 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683): tablet 06225b0fa3414cbbbbf2b907fac36d74 (table test-table [id=fc5ff76055c54312abd4b10cf1cdabf0]) successfully deleted
I20260807 06:12:30.753897 20522 tablet_service.cc:1467] Processing CreateTablet for tablet 0c203ed67d264c6e91f0e6f57f8f5108 (DEFAULT_TABLE table=test-table [id=35d9c5979c3749c893b42b08541ec79d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:30.753911 20731 tablet_service.cc:1467] Processing CreateTablet for tablet 0c203ed67d264c6e91f0e6f57f8f5108 (DEFAULT_TABLE table=test-table [id=35d9c5979c3749c893b42b08541ec79d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:30.754274 20522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c203ed67d264c6e91f0e6f57f8f5108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.754276 20731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c203ed67d264c6e91f0e6f57f8f5108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.757530 21058 tablet_bootstrap.cc:492] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Bootstrap starting.
I20260807 06:12:30.757644 20925 tablet_service.cc:1467] Processing CreateTablet for tablet 0c203ed67d264c6e91f0e6f57f8f5108 (DEFAULT_TABLE table=test-table [id=35d9c5979c3749c893b42b08541ec79d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:30.757949 20925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c203ed67d264c6e91f0e6f57f8f5108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:30.758606 21058 tablet_bootstrap.cc:654] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.760881 21060 tablet_bootstrap.cc:492] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Bootstrap starting.
I20260807 06:12:30.761088 21058 tablet_bootstrap.cc:492] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: No bootstrap required, opened a new log
I20260807 06:12:30.761185 21058 ts_tablet_manager.cc:1397] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:12:30.761806 21060 tablet_bootstrap.cc:654] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.761736 21058 raft_consensus.cc:348] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.761880 21058 raft_consensus.cc:374] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.761924 21058 raft_consensus.cc:729] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8897dbfe3d104807b8169aaa2a73cc7f, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.762081 21058 consensus_queue.cc:260] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.762429 21058 ts_tablet_manager.cc:1428] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 06:12:30.764379 21059 tablet_bootstrap.cc:492] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276: Bootstrap starting.
I20260807 06:12:30.765522 21059 tablet_bootstrap.cc:654] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:30.766971 21060 tablet_bootstrap.cc:492] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: No bootstrap required, opened a new log
I20260807 06:12:30.767098 21060 ts_tablet_manager.cc:1397] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 06:12:30.767622 21060 raft_consensus.cc:348] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.767757 21060 raft_consensus.cc:374] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.767791 21060 raft_consensus.cc:729] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e13573b085646f9a34a548444aae625, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.767946 21060 consensus_queue.cc:260] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.769975 21059 tablet_bootstrap.cc:492] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276: No bootstrap required, opened a new log
I20260807 06:12:30.770073 21059 ts_tablet_manager.cc:1397] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260807 06:12:30.770587 21059 raft_consensus.cc:348] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.770713 21059 raft_consensus.cc:374] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:30.770746 21059 raft_consensus.cc:729] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Initialized, Role: FOLLOWER
I20260807 06:12:30.771373 21060 ts_tablet_manager.cc:1428] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:12:30.773834 21059 consensus_queue.cc:260] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.774200 21059 ts_tablet_manager.cc:1428] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 06:12:30.776144 21186 raft_consensus.cc:482] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:30.776268 21186 raft_consensus.cc:504] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.776767 21186 leader_election.cc:288] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123), 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683)
I20260807 06:12:30.777143 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c203ed67d264c6e91f0e6f57f8f5108" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" is_pre_election: true
I20260807 06:12:30.777304 20761 raft_consensus.cc:2454] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 0.
I20260807 06:12:30.777275 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c203ed67d264c6e91f0e6f57f8f5108" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625" is_pre_election: true
I20260807 06:12:30.777412 20958 raft_consensus.cc:2454] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 0.
I20260807 06:12:30.777638 20453 leader_election.cc:302] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8897dbfe3d104807b8169aaa2a73cc7f, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.777974 21186 raft_consensus.cc:2781] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:30.778034 21186 raft_consensus.cc:482] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:30.778075 21186 raft_consensus.cc:3037] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.779572 21186 raft_consensus.cc:504] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.779990 21186 leader_election.cc:288] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 election: Requested vote from peers 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123), 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683)
I20260807 06:12:30.780380 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c203ed67d264c6e91f0e6f57f8f5108" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e13573b085646f9a34a548444aae625"
I20260807 06:12:30.780517 20958 raft_consensus.cc:3037] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.780617 20761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c203ed67d264c6e91f0e6f57f8f5108" candidate_uuid: "d5e7776bf5684c1cbad259685e719276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8897dbfe3d104807b8169aaa2a73cc7f"
I20260807 06:12:30.780745 20761 raft_consensus.cc:3037] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:30.781827 20958 raft_consensus.cc:2454] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 1.
I20260807 06:12:30.782024 20761 raft_consensus.cc:2454] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5e7776bf5684c1cbad259685e719276 in term 1.
I20260807 06:12:30.782195 20451 leader_election.cc:302] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e13573b085646f9a34a548444aae625, d5e7776bf5684c1cbad259685e719276; no voters:
I20260807 06:12:30.782379 21186 raft_consensus.cc:2781] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:30.782483 21186 raft_consensus.cc:686] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [term 1 LEADER]: Becoming Leader. State: Replica: d5e7776bf5684c1cbad259685e719276, State: Running, Role: LEADER
I20260807 06:12:30.782639 21186 consensus_queue.cc:237] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } }
I20260807 06:12:30.784873 20323 catalog_manager.cc:5495] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5e7776bf5684c1cbad259685e719276 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "d5e7776bf5684c1cbad259685e719276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5e7776bf5684c1cbad259685e719276" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 } health_report { overall_health: UNKNOWN } } }
I20260807 06:12:30.816541 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.812655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:33507 (local address 127.9.133.131:35683)
0807 06:12:30.812795 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.812808 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.812823 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:12:30.812879 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.812882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.812953 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.813031 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:30.813389 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.813937 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.815342 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.815582 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.815676 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.816042 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.816127 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.816185 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.816329 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.816383 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 06:12:30.816596 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.812109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35683 (local address 127.9.133.129:33507)
0807 06:12:30.812545 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.812563 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.812696 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.813072 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.813075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.813085 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:12:30.813264 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.813271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.814079 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.814084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.815189 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.815202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.815710 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.815714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.815822 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.815888 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.816224 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.816232 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.816303 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.816402 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":162,"threads_started":1}
I20260807 06:12:30.823520 20761 raft_consensus.cc:1261] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Refusing update from remote peer d5e7776bf5684c1cbad259685e719276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.823582 20958 raft_consensus.cc:1261] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Refusing update from remote peer d5e7776bf5684c1cbad259685e719276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:30.824007 21186 consensus_queue.cc:1035] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8897dbfe3d104807b8169aaa2a73cc7f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.824237 21177 consensus_queue.cc:1035] T 0c203ed67d264c6e91f0e6f57f8f5108 P d5e7776bf5684c1cbad259685e719276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e13573b085646f9a34a548444aae625" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:30.846248 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.842060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43521 (local address 127.9.133.131:46603)
0807 06:12:30.842352 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.842369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:30.842452 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:30.843040 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:30.843043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:30.843053 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 06:12:30.843180 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.843185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.844006 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.844010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.844797 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:30.844808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.845350 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.845353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:30.845435 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.845471 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.845874 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.845881 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:12:30.845944 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 06:12:30.846046 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260807 06:12:30.846812 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.842603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:46603 (local address 127.9.133.129:43521)
0807 06:12:30.842745 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:30.842751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:30.842771 (+ 20us) server_negotiation.cc:407] Connection header received
0807 06:12:30.842826 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:30.842829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:30.842892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:30.843020 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:30.843296 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.843887 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.844932 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:30.845224 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:30.845341 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:30.845655 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:30.845749 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:30.845820 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:30.846574 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 06:12:30.846663 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:12:30.876065 20322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:30.876308 20322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:30.878974 20322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4882b111e035492f8dd4f5bbfc9f3d57: Sending DeleteTablet for 3 replicas of tablet 0c203ed67d264c6e91f0e6f57f8f5108
I20260807 06:12:30.879854 20731 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c203ed67d264c6e91f0e6f57f8f5108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44060
I20260807 06:12:30.879854 20925 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c203ed67d264c6e91f0e6f57f8f5108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:30 UTC) from {username='oryx', principal='oryx/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:60756
I20260807 06:12:30.880120 21158 tablet_replica.cc:330] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: stopping tablet replica
I20260807 06:12:30.880120 21155 tablet_replica.cc:330] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: stopping tablet replica
I20260807 06:12:30.880281 21155 raft_consensus.cc:2229] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:30.880281 21158 raft_consensus.cc:2229] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:30.880420 21155 raft_consensus.cc:2258] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.880420 21158 raft_consensus.cc:2258] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:30.881153 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 20424
I20260807 06:12:30.881366 21158 ts_tablet_manager.cc:1905] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.885392 21158 ts_tablet_manager.cc:1918] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:12:30.885488 21158 log.cc:1196] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/0c203ed67d264c6e91f0e6f57f8f5108
I20260807 06:12:30.885761 21158 ts_tablet_manager.cc:1939] T 0c203ed67d264c6e91f0e6f57f8f5108 P 8897dbfe3d104807b8169aaa2a73cc7f: Deleting consensus metadata
I20260807 06:12:30.887454 20299 catalog_manager.cc:4841] TS 8897dbfe3d104807b8169aaa2a73cc7f (127.9.133.130:38123): tablet 0c203ed67d264c6e91f0e6f57f8f5108 (table test-table [id=35d9c5979c3749c893b42b08541ec79d]) successfully deleted
I20260807 06:12:30.890379 21155 ts_tablet_manager.cc:1905] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:30.894213 21155 ts_tablet_manager.cc:1918] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 06:12:30.894333 21155 log.cc:1196] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/0c203ed67d264c6e91f0e6f57f8f5108
I20260807 06:12:30.894594 21155 ts_tablet_manager.cc:1939] T 0c203ed67d264c6e91f0e6f57f8f5108 P 9e13573b085646f9a34a548444aae625: Deleting consensus metadata
I20260807 06:12:30.895529 20297 catalog_manager.cc:4841] TS 9e13573b085646f9a34a548444aae625 (127.9.133.131:35683): tablet 0c203ed67d264c6e91f0e6f57f8f5108 (table test-table [id=35d9c5979c3749c893b42b08541ec79d]) successfully deleted
W20260807 06:12:30.902109 20297 connection.cc:535] client connection to 127.9.133.129:43521 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 06:12:30.902364 20297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:12:30.902632 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 20638
I20260807 06:12:30.913808 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:30.905333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:43521 (local address 127.0.0.1:44132)
0807 06:12:30.913459 (+ 8126us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:30.913612 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7969,"thread_start_us":997,"threads_started":1}
W20260807 06:12:30.914076 20297 catalog_manager.cc:4584] TS d5e7776bf5684c1cbad259685e719276 (127.9.133.129:43521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c203ed67d264c6e91f0e6f57f8f5108: Network error: Client connection negotiation failed: client connection to 127.9.133.129:43521: connect: Connection refused (error 111)
I20260807 06:12:30.922103 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 20834
I20260807 06:12:30.939469 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 20276
2026-08-07T06:12:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4010 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:31 dist-test-slave-h3ft krb5kdc[21210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:31 dist-test-slave-h3ft krb5kdc[21210](info): set up 2 sockets
Aug 07 06:12:31 dist-test-slave-h3ft krb5kdc[21210](info): commencing operation
krb5kdc: starting...
W20260807 06:12:32.215152 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:32 dist-test-slave-h3ft krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:32.366384 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39783
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:46747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:39783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:32.502847 21230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:32.503305 21230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:32.503383 21230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:32.507568 21230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:32.507675 21230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:32.507711 21230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:32.507752 21230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:32.507787 21230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:32.513674 21230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:46747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:39783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:32.515154 21230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:32.516497 21230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:32.526858 21236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:32.526859 21238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:32.526976 21235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:32.527213 21230 server_base.cc:1034] running on GCE node
I20260807 06:12:32.528093 21230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:32.529686 21230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:32.530941 21230 hybrid_clock.cc:648] HybridClock initialized: now 1786083152530889 us; error 72 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T06:12:32Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1608 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:32 dist-test-slave-h3ft krb5kdc[21246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:32 dist-test-slave-h3ft krb5kdc[21246](info): set up 2 sockets
Aug 07 06:12:32 dist-test-slave-h3ft krb5kdc[21246](info): commencing operation
krb5kdc: starting...
W20260807 06:12:33.855744 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:33 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:34.008675 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36533
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:36533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:34.145967 21266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:34.146310 21266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:34.146355 21266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:34.150490 21266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:34.150563 21266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:34.150583 21266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:34.150599 21266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:34.150615 21266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:34.156236 21266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:36533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:34.157563 21266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:34.158800 21266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:34.169101 21276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:34.169107 21280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:34.169193 21278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:34.169394 21266 server_base.cc:1034] running on GCE node
I20260807 06:12:34.170207 21266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:34.171653 21266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:34.172832 21266 hybrid_clock.cc:648] HybridClock initialized: now 1786083154172800 us; error 51 us; skew 500 ppm
Aug 07 06:12:34 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:34.176127 21266 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:34.177239 21266 webserver.cc:460] Webserver started at http://127.9.133.190:46551/ using document root <none> and password file <none>
I20260807 06:12:34.177948 21266 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:34.178049 21266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:34.178328 21266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:34.180889 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "f632aa9cb3f74816b9895374335a15b6"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "6930eba67ebb557e40990ef305cca403"
server_key_iv: "3ba1b3bc3c4ea43cc2382b5c2a39370d"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.181571 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "f632aa9cb3f74816b9895374335a15b6"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "6930eba67ebb557e40990ef305cca403"
server_key_iv: "3ba1b3bc3c4ea43cc2382b5c2a39370d"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.186141 21266 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:34.189083 21290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.190641 21266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:34.190898 21266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "f632aa9cb3f74816b9895374335a15b6"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "6930eba67ebb557e40990ef305cca403"
server_key_iv: "3ba1b3bc3c4ea43cc2382b5c2a39370d"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.191080 21266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:34.224301 21266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:34.228415 21266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:34.228716 21266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:34.237401 21266 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:36533
I20260807 06:12:34.237427 21365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:36533 every 8 connection(s)
I20260807 06:12:34.238626 21266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:34.242069 21366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:34.248634 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 21266
I20260807 06:12:34.248806 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:34.249151 21366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Bootstrap starting.
I20260807 06:12:34.249207 9750 external_mini_cluster.cc:1428] Setting key 431ac18c54917f546ab324d92fe68e29
I20260807 06:12:34.252305 21366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:34.253203 21366 log.cc:824] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:34.255434 21366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: No bootstrap required, opened a new log
Aug 07 06:12:34 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:34.259925 21366 raft_consensus.cc:348] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:34.260197 21366 raft_consensus.cc:374] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:34.260272 21366 raft_consensus.cc:729] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f632aa9cb3f74816b9895374335a15b6, State: Initialized, Role: FOLLOWER
I20260807 06:12:34.260828 21366 consensus_queue.cc:260] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:34.261039 21366 raft_consensus.cc:388] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:34.261145 21366 raft_consensus.cc:482] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:34.261260 21366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:34.262399 21366 raft_consensus.cc:504] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:34.262812 21366 leader_election.cc:302] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f632aa9cb3f74816b9895374335a15b6; no voters:
I20260807 06:12:34.263293 21366 leader_election.cc:288] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:34.263376 21373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:34.263645 21373 raft_consensus.cc:686] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 LEADER]: Becoming Leader. State: Replica: f632aa9cb3f74816b9895374335a15b6, State: Running, Role: LEADER
I20260807 06:12:34.263991 21373 consensus_queue.cc:237] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:34.264909 21366 sys_catalog.cc:564] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:34.265877 21375 sys_catalog.cc:455] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f632aa9cb3f74816b9895374335a15b6. Latest consensus state: current_term: 1 leader_uuid: "f632aa9cb3f74816b9895374335a15b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } } }
I20260807 06:12:34.266021 21375 sys_catalog.cc:458] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:34.266383 21374 sys_catalog.cc:455] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f632aa9cb3f74816b9895374335a15b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } } }
I20260807 06:12:34.266476 21374 sys_catalog.cc:458] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:34.266769 21389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:34.266746 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:34.250974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.9.133.190:36533)
0807 06:12:34.251530 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:34.251548 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:34.251591 (+ 43us) server_negotiation.cc:407] Connection header received
0807 06:12:34.252340 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:34.252389 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:34.252825 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:34.253232 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:34.254205 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.255365 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.256349 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.256874 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.257051 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:34.260233 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:34.260240 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:34.260279 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:34.260332 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:34.262453 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:34.262985 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:34.262993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:34.263045 (+ 52us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:34.263189 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:34.263521 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:34.263525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:34.263527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:34.264293 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:34.264528 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:34.264867 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 06:12:34.265302 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":162,"threads_started":1}
I20260807 06:12:34.270457 21389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:34.277081 21389 catalog_manager.cc:1347] Generated new cluster ID: 682d49b8d9ef44e4898a7d6443885b46
I20260807 06:12:34.277158 21389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:34.293638 21389 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:34.295183 21389 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:34.303356 21389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Generated new TSK 0
I20260807 06:12:34.304351 21389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:34.348767 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36533
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:34.484412 21406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:34.484771 21406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:34.484819 21406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:34.489152 21406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:34.489229 21406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:34.489339 21406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:34.495304 21406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:34.496897 21406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:34.498209 21406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:34.510035 21413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:34.510027 21412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:34.510021 21415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:34.510301 21406 server_base.cc:1034] running on GCE node
I20260807 06:12:34.510859 21406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:34.511677 21406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:34.512859 21406 hybrid_clock.cc:648] HybridClock initialized: now 1786083154512822 us; error 50 us; skew 500 ppm
Aug 07 06:12:34 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:34.516300 21406 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:34.517553 21406 webserver.cc:460] Webserver started at http://127.9.133.129:46005/ using document root <none> and password file <none>
I20260807 06:12:34.518292 21406 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:34.518396 21406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:34.518680 21406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:34.521364 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "d56082808c244f2693e9338c34c59519"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "f44fa985a68d03453686ae9b108994d2"
server_key_iv: "d6be6a99fcd45c45545ff92f784392e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.522058 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "d56082808c244f2693e9338c34c59519"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "f44fa985a68d03453686ae9b108994d2"
server_key_iv: "d6be6a99fcd45c45545ff92f784392e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.527464 21406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 06:12:34.530599 21426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.532230 21406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:34.532405 21406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "d56082808c244f2693e9338c34c59519"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "f44fa985a68d03453686ae9b108994d2"
server_key_iv: "d6be6a99fcd45c45545ff92f784392e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.532552 21406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:34.552562 21406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:34.556321 21406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:34.556587 21406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:34.557379 21406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:34.558439 21406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:34.558518 21406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.558596 21406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:34.558650 21406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.569923 21406 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:33015
I20260807 06:12:34.569952 21587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:33015 every 8 connection(s)
I20260807 06:12:34.571149 21406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:34.578334 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 21406
I20260807 06:12:34.578519 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:34.578913 9750 external_mini_cluster.cc:1428] Setting key de6583af8ca7296f1cac84b13aa3bef8
Aug 07 06:12:34 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:34.588919 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:34.573472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:57595 (local address 127.9.133.190:36533)
0807 06:12:34.573645 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:34.573649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:34.574864 (+ 1215us) server_negotiation.cc:407] Connection header received
0807 06:12:34.576032 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:34.576036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:34.576110 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:34.576225 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:34.577640 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.578321 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.579879 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.580143 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.580306 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:34.584035 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:34.584041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:34.584046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:34.584091 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:34.586053 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:34.586668 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:34.586675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:34.586680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:34.586797 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:34.587243 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:34.587250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:34.587254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:34.587617 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:34.587745 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:34.588421 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 06:12:34.588617 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":58}
I20260807 06:12:34.589850 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:34.573875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:57595)
0807 06:12:34.574681 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:34.574732 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:34.575785 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:34.576431 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:34.576439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:34.576870 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:34.577465 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:34.577481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.578472 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.578476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:34.579749 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:34.579760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.580332 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.580337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:34.580604 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:34.581443 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:34.581462 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:34.583777 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:34.586202 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:34.586207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:34.586225 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:34.586542 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:34.586894 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:34.586896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:34.586898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:34.587131 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:34.587756 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:34.587765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:34.588117 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 06:12:34.588305 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 06:12:34.588657 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":140,"threads_started":1}
I20260807 06:12:34.591517 21588 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36533
I20260807 06:12:34.591873 21588 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:34.592725 21588 heartbeater.cc:511] Master 127.9.133.190:36533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:34.595036 21315 ts_manager.cc:194] Registered new tserver with Master: d56082808c244f2693e9338c34c59519 (127.9.133.129:33015)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:34.596715 21315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:57595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:34.614501 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36533
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:34.750650 21597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:34.751096 21597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:34.751206 21597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:34.755676 21597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:34.755770 21597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:34.755863 21597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:34.761660 21597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:34.763154 21597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:34.764480 21597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:34.776455 21603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:34.776528 21602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:34.776455 21605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:34.776700 21597 server_base.cc:1034] running on GCE node
I20260807 06:12:34.777290 21597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:34.778072 21597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:34.779281 21597 hybrid_clock.cc:648] HybridClock initialized: now 1786083154779263 us; error 30 us; skew 500 ppm
Aug 07 06:12:34 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:34.782662 21597 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:34.783834 21597 webserver.cc:460] Webserver started at http://127.9.133.130:45365/ using document root <none> and password file <none>
I20260807 06:12:34.784566 21597 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:34.784667 21597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:34.784948 21597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:34.787523 21597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "0baf178dae23436aa24781dde518f919"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "1d8eb9d96e1a972d5f55fdb58563a76b"
server_key_iv: "e854bea10700bb2e667e2f0f8c86576c"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.788182 21597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "0baf178dae23436aa24781dde518f919"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "1d8eb9d96e1a972d5f55fdb58563a76b"
server_key_iv: "e854bea10700bb2e667e2f0f8c86576c"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.792768 21597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:34.795775 21617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.797375 21597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:34.797535 21597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "0baf178dae23436aa24781dde518f919"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "1d8eb9d96e1a972d5f55fdb58563a76b"
server_key_iv: "e854bea10700bb2e667e2f0f8c86576c"
server_key_version: "encryptionkey@0"
I20260807 06:12:34.797672 21597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:34.810644 21597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:34.814881 21597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:34.815165 21597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:34.815915 21597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:34.816951 21597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:34.817036 21597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.817116 21597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:34.817160 21597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:34.828650 21597 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:46573
I20260807 06:12:34.828671 21785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:46573 every 8 connection(s)
I20260807 06:12:34.829847 21597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:34.834091 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 21597
I20260807 06:12:34.834252 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:34.834662 9750 external_mini_cluster.cc:1428] Setting key 37a493f34430bd07757fd79faf498d41
Aug 07 06:12:34 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:12:34.850632 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:34.832214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34983 (local address 127.9.133.190:36533)
0807 06:12:34.832418 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:34.832423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:34.833760 (+ 1337us) server_negotiation.cc:407] Connection header received
0807 06:12:34.835249 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:34.835253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:34.835331 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:34.835431 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:34.837263 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.837928 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.839793 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.840222 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.840351 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:34.844028 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:34.844035 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:34.844041 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:34.844119 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:34.847332 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:34.848272 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:34.848278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:34.848280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:34.848410 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:34.848951 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:34.848955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:34.848957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:34.849265 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:34.849542 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:34.850256 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 06:12:34.850415 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:12:34.851910 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:34.832749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:34983)
0807 06:12:34.833556 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:34.833605 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:34.834905 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:34.835667 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:34.835680 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:34.836141 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:34.837086 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:34.837112 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.838065 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.838071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:34.839666 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:34.839677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:34.840397 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:34.840400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:34.840559 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:34.841340 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:34.841368 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:34.843767 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:34.847529 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:34.847537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:34.847558 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:34.848045 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:34.848502 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:34.848506 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:34.848509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:34.848736 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:34.849536 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:34.849544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:34.849969 (+ 425us) client_negotiation.cc:769] Sending connection context
0807 06:12:34.850196 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 06:12:34.850609 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":139,"threads_started":1}
I20260807 06:12:34.855067 21786 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36533
I20260807 06:12:34.855427 21786 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:34.856151 21786 heartbeater.cc:511] Master 127.9.133.190:36533 requested a full tablet report, sending...
I20260807 06:12:34.857692 21315 ts_manager.cc:194] Registered new tserver with Master: 0baf178dae23436aa24781dde518f919 (127.9.133.130:46573)
I20260807 06:12:34.858397 21315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:34983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:34.871389 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36533
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:35.010617 21793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:35.011139 21793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:35.011238 21793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:35.015604 21793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:35.015729 21793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:35.015837 21793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:35.021979 21793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:35.023538 21793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:35.024919 21793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:35.036489 21802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:35.036509 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:35.036523 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:35.036993 21793 server_base.cc:1034] running on GCE node
I20260807 06:12:35.037557 21793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:35.038319 21793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:35.039521 21793 hybrid_clock.cc:648] HybridClock initialized: now 1786083155039490 us; error 58 us; skew 500 ppm
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:35.042749 21793 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:35.043923 21793 webserver.cc:460] Webserver started at http://127.9.133.131:43925/ using document root <none> and password file <none>
I20260807 06:12:35.044608 21793 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:35.044708 21793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:35.044986 21793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:35.047457 21793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "3b4ae40cd6754c9987385c00d5af5773"
format_stamp: "Formatted at 2026-08-07 06:12:35 on dist-test-slave-h3ft"
server_key: "082b8b51825be05b77e6db07a2b56c99"
server_key_iv: "325c7ef708b9844a942dd542de129d72"
server_key_version: "encryptionkey@0"
I20260807 06:12:35.048110 21793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "3b4ae40cd6754c9987385c00d5af5773"
format_stamp: "Formatted at 2026-08-07 06:12:35 on dist-test-slave-h3ft"
server_key: "082b8b51825be05b77e6db07a2b56c99"
server_key_iv: "325c7ef708b9844a942dd542de129d72"
server_key_version: "encryptionkey@0"
I20260807 06:12:35.052609 21793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:35.055562 21813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:35.057019 21793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:35.057173 21793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "3b4ae40cd6754c9987385c00d5af5773"
format_stamp: "Formatted at 2026-08-07 06:12:35 on dist-test-slave-h3ft"
server_key: "082b8b51825be05b77e6db07a2b56c99"
server_key_iv: "325c7ef708b9844a942dd542de129d72"
server_key_version: "encryptionkey@0"
I20260807 06:12:35.057302 21793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:35.070719 21793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:35.075147 21793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:35.075431 21793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:35.076287 21793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:35.077373 21793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:35.077452 21793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:35.077538 21793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:35.077590 21793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:35.089052 21793 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:35349
I20260807 06:12:35.089083 21975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:35349 every 8 connection(s)
I20260807 06:12:35.090289 21793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:35.090821 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 21793
I20260807 06:12:35.090993 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:35.091457 9750 external_mini_cluster.cc:1428] Setting key 2201a17ba871ca715dccf12d889f46b3
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:35.109045 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:35.093158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:53733 (local address 127.9.133.190:36533)
0807 06:12:35.093333 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:35.093339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:35.095107 (+ 1768us) server_negotiation.cc:407] Connection header received
0807 06:12:35.096341 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:35.096345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:35.096425 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:35.096517 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:35.098104 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.098731 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.100270 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.100538 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.100652 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.104343 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:35.104347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:35.104350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:35.104384 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:35.106317 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:35.107129 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:35.107133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:35.107134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:35.107225 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:35.107643 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:35.107647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:35.107649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:35.107890 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:35.108011 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:35.108682 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 06:12:35.108840 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:12:35.110291 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:35.093866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:53733)
0807 06:12:35.094898 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:35.094952 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:35.096087 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:35.096761 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:35.096770 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:35.097263 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:35.097925 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.097943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.098899 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.098903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.100097 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.100109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.100746 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.100748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.100948 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.101862 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:35.101882 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:35.104066 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:35.106459 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:35.106468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:35.106499 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:35.106966 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:35.107327 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:35.107331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:35.107333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:35.107530 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:35.108016 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:35.108027 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:35.108390 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 06:12:35.108627 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 06:12:35.109099 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":174,"threads_started":1}
I20260807 06:12:35.111846 21976 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36533
I20260807 06:12:35.112174 21976 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:35.112866 21976 heartbeater.cc:511] Master 127.9.133.190:36533 requested a full tablet report, sending...
I20260807 06:12:35.114256 21315 ts_manager.cc:194] Registered new tserver with Master: 3b4ae40cd6754c9987385c00d5af5773 (127.9.133.131:35349)
I20260807 06:12:35.114977 21315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:53733
I20260807 06:12:35.119745 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab.
Entry for principal oryx/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab.
WARNING: no policy specified for oryx/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab.
Entry for principal oryx/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab.
WARNING: no policy specified for oryx/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab.
Entry for principal oryx/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab.
I20260807 06:12:35.218717 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 21406
I20260807 06:12:35.231303 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 21597
I20260807 06:12:35.248503 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 21793
I20260807 06:12:35.259688 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 21266
I20260807 06:12:35.275969 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36533
--webserver_interface=127.9.133.190
--webserver_port=46551
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:36533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:35.409199 21990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:35.409523 21990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:35.409597 21990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:35.413901 21990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:35.413985 21990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:35.414005 21990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:35.414022 21990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:35.414064 21990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:35.420077 21990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:36533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:35.421483 21990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:35.422811 21990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:35.433207 21999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:35.433267 21996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:35.433499 21990 server_base.cc:1034] running on GCE node
W20260807 06:12:35.433640 21997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:35.434412 21990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:35.435972 21990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:35.437157 21990 hybrid_clock.cc:648] HybridClock initialized: now 1786083155437145 us; error 29 us; skew 500 ppm
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:35.440856 21990 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:35.442405 21990 webserver.cc:460] Webserver started at http://127.9.133.190:46551/ using document root <none> and password file <none>
I20260807 06:12:35.443239 21990 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:35.443385 21990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:35.448799 21990 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 06:12:35.451488 22009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:35.452914 21990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:35.453086 21990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "f632aa9cb3f74816b9895374335a15b6"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "6930eba67ebb557e40990ef305cca403"
server_key_iv: "3ba1b3bc3c4ea43cc2382b5c2a39370d"
server_key_version: "encryptionkey@0"
I20260807 06:12:35.453766 21990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:35.471530 21990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:35.476001 21990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:35.476315 21990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:35.485679 21990 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:36533
I20260807 06:12:35.485703 22084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:36533 every 8 connection(s)
I20260807 06:12:35.487071 21990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:35.492551 22086 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:12:35.494539 22086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Bootstrap starting.
I20260807 06:12:35.495115 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 21990
I20260807 06:12:35.495661 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:33015
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:36533
--webserver_port=46005
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:12:35.515332 22086 log.cc:824] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:35.524616 22086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:12:35.525121 22086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Bootstrap complete.
I20260807 06:12:35.530264 22086 raft_consensus.cc:348] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:35.530799 22086 raft_consensus.cc:729] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f632aa9cb3f74816b9895374335a15b6, State: Initialized, Role: FOLLOWER
I20260807 06:12:35.531527 22086 consensus_queue.cc:260] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:35.531709 22086 raft_consensus.cc:388] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:35.531810 22086 raft_consensus.cc:482] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:35.531944 22086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:12:35.534402 22086 raft_consensus.cc:504] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:35.534840 22086 leader_election.cc:302] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f632aa9cb3f74816b9895374335a15b6; no voters:
I20260807 06:12:35.535306 22086 leader_election.cc:288] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:12:35.535441 22091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:12:35.535696 22091 raft_consensus.cc:686] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [term 2 LEADER]: Becoming Leader. State: Replica: f632aa9cb3f74816b9895374335a15b6, State: Running, Role: LEADER
I20260807 06:12:35.536067 22091 consensus_queue.cc:237] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } }
I20260807 06:12:35.536902 22086 sys_catalog.cc:564] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:35.537884 22092 sys_catalog.cc:455] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f632aa9cb3f74816b9895374335a15b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } } }
I20260807 06:12:35.538100 22092 sys_catalog.cc:458] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:35.537881 22093 sys_catalog.cc:455] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f632aa9cb3f74816b9895374335a15b6. Latest consensus state: current_term: 2 leader_uuid: "f632aa9cb3f74816b9895374335a15b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f632aa9cb3f74816b9895374335a15b6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36533 } } }
I20260807 06:12:35.538225 22093 sys_catalog.cc:458] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:35.538558 22102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:35.542572 22102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:35.543308 22102 catalog_manager.cc:1259] Loaded cluster ID: 682d49b8d9ef44e4898a7d6443885b46
I20260807 06:12:35.543373 22102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:35.545908 22102 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:35.546627 22102 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f632aa9cb3f74816b9895374335a15b6: Loaded TSK: 0
I20260807 06:12:35.547993 22102 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:12:35.681794 22089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:35.682781 22089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:35.683063 22089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:35.693002 22089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:35.693230 22089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:35.693584 22089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:35.708688 22089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.129.keytab
--principal=oryx/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:33015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=46005
--tserver_master_addrs=127.9.133.190:36533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:35.712433 22089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:35.715811 22089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:35.741793 22119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:35.741842 22120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:35.742017 22124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:35.743273 22089 server_base.cc:1034] running on GCE node
I20260807 06:12:35.744292 22089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:35.745920 22089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:35.749293 22089 hybrid_clock.cc:648] HybridClock initialized: now 1786083155749213 us; error 114 us; skew 500 ppm
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083155, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:35.756754 22089 init.cc:377] Logged in from keytab as oryx/127.9.133.129@KRBTEST.COM (short username oryx)
I20260807 06:12:35.759793 22089 webserver.cc:460] Webserver started at http://127.9.133.129:46005/ using document root <none> and password file <none>
I20260807 06:12:35.761058 22089 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:35.761209 22089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:35.772318 22089 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.006s sys 0.000s
I20260807 06:12:35.778115 22137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:35.781975 22089 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.000s
I20260807 06:12:35.782232 22089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "d56082808c244f2693e9338c34c59519"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "f44fa985a68d03453686ae9b108994d2"
server_key_iv: "d6be6a99fcd45c45545ff92f784392e5"
server_key_version: "encryptionkey@0"
I20260807 06:12:35.783766 22089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:35.824162 22089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:35.833633 22089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:35.834257 22089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:35.836042 22089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:35.838680 22089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:35.838796 22089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:35.839038 22089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:35.839176 22089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 06:12:35.864761 22089 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:33015
I20260807 06:12:35.864846 22307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:33015 every 8 connection(s)
I20260807 06:12:35.867558 22089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:35.871910 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 22089
I20260807 06:12:35.873174 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:46573
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:36533
--webserver_port=45365
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:12:35.902570 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:35.873016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:41395 (local address 127.9.133.190:36533)
0807 06:12:35.874386 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:35.874470 (+ 84us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:35.876051 (+ 1581us) server_negotiation.cc:407] Connection header received
0807 06:12:35.879092 (+ 3041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:35.879126 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:35.880042 (+ 916us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:35.881426 (+ 1384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:35.885359 (+ 3933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.887339 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.889927 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.891135 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.891599 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.900186 (+ 8587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:41395: BlockingRecv error: recv got EOF from 127.9.133.129:41395 (error 108)
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":370,"threads_started":1}
I20260807 06:12:35.902575 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:35.873688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:41395)
0807 06:12:35.875590 (+ 1902us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:35.875730 (+ 140us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:35.877883 (+ 2153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:35.881705 (+ 3822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:35.881737 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:35.883717 (+ 1980us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:35.884947 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.884991 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.887723 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.887735 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.889627 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.889658 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.891562 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.891568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.891990 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.893418 (+ 1428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:35.893460 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:35.899496 (+ 6036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1390,"thread_start_us":418,"threads_started":1}
W20260807 06:12:35.904385 22308 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:36533 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.133.190:36533: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:35.917127 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:35.905490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:47103)
0807 06:12:35.905741 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:35.905782 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:35.905974 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:35.906641 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:35.906651 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:35.906977 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:35.907291 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.907357 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.909482 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.909485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.910458 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.910515 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.911585 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.911589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.911737 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.912814 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:35.912845 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:35.916864 (+ 4019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 06:12:35.917428 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:35.905635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:47103 (local address 127.9.133.190:36533)
0807 06:12:35.905935 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:35.905951 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:35.906029 (+ 78us) server_negotiation.cc:407] Connection header received
0807 06:12:35.906217 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:35.906221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:35.906371 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:35.906593 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:35.907582 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.909258 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.910685 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.911294 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.911567 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.917276 (+ 5709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:47103: BlockingRecv error: recv got EOF from 127.9.133.129:47103 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:35 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:35.930672 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:35.918700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:34499)
0807 06:12:35.919073 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:35.919112 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:35.919273 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:35.919781 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:35.919784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:35.920332 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:35.920605 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.920627 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.922557 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.922566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.923660 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:35.923684 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.924899 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.924902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:35.925026 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.926016 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:35.926063 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:35.930345 (+ 4282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144}
W20260807 06:12:35.931382 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:35.918876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:34499 (local address 127.9.133.190:36533)
0807 06:12:35.919225 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:35.919230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:35.919258 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:12:35.919429 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:35.919434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:35.919579 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:35.919771 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:35.920871 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.922315 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.923879 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:35.924634 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:35.924920 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:35.930871 (+ 5951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:34499: BlockingRecv error: recv got EOF from 127.9.133.129:34499 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
W20260807 06:12:35.931823 22308 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:12:36.060137 22313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:36.060497 22313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:36.060544 22313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:36.064759 22313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:36.064848 22313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:36.064913 22313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:36.070745 22313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.130.keytab
--principal=oryx/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:46573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=45365
--tserver_master_addrs=127.9.133.190:36533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:36.072229 22313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:36.073566 22313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:36.085956 22322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:36.086081 22320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:36.086365 22326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:36.086663 22313 server_base.cc:1034] running on GCE node
I20260807 06:12:36.087294 22313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:36.088197 22313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:36.089389 22313 hybrid_clock.cc:648] HybridClock initialized: now 1786083156089335 us; error 66 us; skew 500 ppm
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083156, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:36.093174 22313 init.cc:377] Logged in from keytab as oryx/127.9.133.130@KRBTEST.COM (short username oryx)
I20260807 06:12:36.094374 22313 webserver.cc:460] Webserver started at http://127.9.133.130:45365/ using document root <none> and password file <none>
I20260807 06:12:36.095202 22313 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:36.095306 22313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:36.100701 22313 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:36.103528 22336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:36.105167 22313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 06:12:36.105331 22313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "0baf178dae23436aa24781dde518f919"
format_stamp: "Formatted at 2026-08-07 06:12:34 on dist-test-slave-h3ft"
server_key: "1d8eb9d96e1a972d5f55fdb58563a76b"
server_key_iv: "e854bea10700bb2e667e2f0f8c86576c"
server_key_version: "encryptionkey@0"
I20260807 06:12:36.106068 22313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:36.118866 22313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:36.122967 22313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:36.123267 22313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:36.124060 22313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:36.125123 22313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:36.125175 22313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:36.125252 22313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:36.125285 22313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:36.136561 22313 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:46573
I20260807 06:12:36.136581 22505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:46573 every 8 connection(s)
I20260807 06:12:36.138033 22313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:36.141517 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 22313
I20260807 06:12:36.142221 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:35349
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:36533
--webserver_port=43925
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:12:36.151963 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.140538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:40713 (local address 127.9.133.190:36533)
0807 06:12:36.140718 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.140722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.142240 (+ 1518us) server_negotiation.cc:407] Connection header received
0807 06:12:36.143459 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.143464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.143543 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.143655 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.145093 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.145690 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.146978 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.147281 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.147408 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.151794 (+ 4386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:40713: BlockingRecv error: recv got EOF from 127.9.133.130:40713 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 06:12:36.152806 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.141042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:40713)
0807 06:12:36.142051 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.142102 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.143215 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.143873 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.143882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.144313 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.144919 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.144935 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.145839 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.145844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.146819 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.146829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.147461 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.147467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.147676 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.148412 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.148430 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.151638 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":225,"threads_started":1}
W20260807 06:12:36.153422 22506 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:36533 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.133.190:36533: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:36.160475 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.154036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:43769)
0807 06:12:36.154166 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.154184 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.154274 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.154628 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.154630 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.154780 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.154883 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.154887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.156079 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.156081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.156626 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.156634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.157443 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.157446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.157523 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.158110 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.158123 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.160357 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 06:12:36.160705 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.154143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:43769 (local address 127.9.133.190:36533)
0807 06:12:36.154327 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.154332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.154355 (+ 23us) server_negotiation.cc:407] Connection header received
0807 06:12:36.154423 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.154428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.154502 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.154620 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.155037 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.155948 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.156755 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.157306 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.157451 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.160552 (+ 3101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:43769: BlockingRecv error: recv got EOF from 127.9.133.130:43769 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:36.167133 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.161269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:44893)
0807 06:12:36.161390 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.161406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.161491 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.161758 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.161760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.161891 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.161965 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.161969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.162989 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.162991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.163577 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.163585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.164265 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.164268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.164346 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.164927 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.164940 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.166925 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 06:12:36.167286 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.161353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:44893 (local address 127.9.133.190:36533)
0807 06:12:36.161478 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.161483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.161501 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:12:36.161587 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.161590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.161641 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.161726 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.162068 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.162884 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.163705 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.164116 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.164244 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.167140 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:44893: BlockingRecv error: recv got EOF from 127.9.133.130:44893 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 06:12:36.167364 22506 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:12:36.283392 22513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:36.283702 22513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:36.283753 22513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:36.287910 22513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:36.287986 22513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:36.288053 22513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:36.293776 22513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/krb5kdc/oryx_127.9.133.131.keytab
--principal=oryx/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:35349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=43925
--tserver_master_addrs=127.9.133.190:36533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:36.295184 22513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:36.296550 22513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:36.307904 22528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:36.307929 22526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:36.307849 22525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:36.308391 22513 server_base.cc:1034] running on GCE node
I20260807 06:12:36.308866 22513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:36.309648 22513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:36.310873 22513 hybrid_clock.cc:648] HybridClock initialized: now 1786083156310851 us; error 42 us; skew 500 ppm
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083156, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:36.314261 22513 init.cc:377] Logged in from keytab as oryx/127.9.133.131@KRBTEST.COM (short username oryx)
I20260807 06:12:36.315385 22513 webserver.cc:460] Webserver started at http://127.9.133.131:43925/ using document root <none> and password file <none>
I20260807 06:12:36.316066 22513 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:36.316154 22513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:36.320806 22513 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:36.323542 22536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:36.325091 22513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 06:12:36.325209 22513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "3b4ae40cd6754c9987385c00d5af5773"
format_stamp: "Formatted at 2026-08-07 06:12:35 on dist-test-slave-h3ft"
server_key: "082b8b51825be05b77e6db07a2b56c99"
server_key_iv: "325c7ef708b9844a942dd542de129d72"
server_key_version: "encryptionkey@0"
I20260807 06:12:36.325860 22513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:36.340009 22513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:36.343737 22513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:36.343935 22513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:36.344651 22513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:36.345639 22513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:36.345692 22513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:36.345738 22513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:36.345757 22513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:36.356675 22513 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:35349
I20260807 06:12:36.356698 22720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:35349 every 8 connection(s)
I20260807 06:12:36.358021 22513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:36.362478 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 22513
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 06:12:36.371304 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.360304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:34373 (local address 127.9.133.190:36533)
0807 06:12:36.360540 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.360546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.361777 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 06:12:36.363107 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.363114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.363195 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.363295 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.364940 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.365515 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.366760 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.366978 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.367120 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.371126 (+ 4006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:34373: BlockingRecv error: recv got EOF from 127.9.133.131:34373 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 06:12:36.372284 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.360824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:34373)
0807 06:12:36.361608 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.361656 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.362814 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.363538 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.363547 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.364081 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.364781 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.364797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.365649 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.365653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.366620 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.366629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.367175 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.367180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.367413 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.368024 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.368042 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.370969 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":136,"threads_started":1}
W20260807 06:12:36.372920 22721 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:36533 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.133.190:36533: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
I20260807 06:12:36.374071 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.363811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.9.133.190:36533)
0807 06:12:36.364131 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.364136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.364150 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:12:36.364193 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.364196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.364238 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.364342 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.365148 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.365698 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.366639 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.366876 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.366987 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.368697 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:36.368707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:36.368729 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:36.368764 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:36.371092 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:36.371636 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:36.371643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:36.371649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:36.371835 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:36.372115 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:36.372118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:36.372120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:36.372675 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:36.372922 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:36.373570 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 06:12:36.373865 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":118,"threads_started":1}
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:36.378839 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.373471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:56483)
0807 06:12:36.373656 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.373670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.373752 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.374002 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.374005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.374138 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.374282 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.374289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.374994 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.374997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.375622 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.375632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.376173 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.376177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.376270 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.376834 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.376851 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.378742 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260807 06:12:36.379055 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.373559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:56483 (local address 127.9.133.190:36533)
0807 06:12:36.373684 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.373688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.373719 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:12:36.373833 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.373837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.373880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.373961 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.374382 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.374879 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.375735 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.376048 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.376169 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.378866 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:56483: BlockingRecv error: recv got EOF from 127.9.133.131:56483 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:36.384281 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.379463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:49963)
0807 06:12:36.379592 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.379607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.379685 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.379913 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.379916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.380076 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.380145 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.380150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.380860 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.380865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.381380 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.381388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.381829 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.381833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.381913 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.382396 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.382408 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.384134 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 06:12:36.384356 22310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.379567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:49963 (local address 127.9.133.190:36533)
0807 06:12:36.379710 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.379716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.379732 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:12:36.379769 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.379772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.379813 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.379896 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.380257 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.380751 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.381502 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.381717 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.381829 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.384239 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:49963: BlockingRecv error: recv got EOF from 127.9.133.131:49963 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 06:12:36.384541 22721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:36 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:36.940413 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:36.932907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:45431)
0807 06:12:36.933284 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:36.933302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:36.933448 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:36.933830 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:36.933833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:36.934124 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:36.934255 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.934261 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.935206 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.935209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.935969 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:36.935991 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.936532 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.936536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:36.936617 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.937275 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:36.937289 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:36.940174 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":122,"threads_started":1}
W20260807 06:12:36.940526 22729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:36.933000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:45431 (local address 127.9.133.190:36533)
0807 06:12:36.933386 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:36.933390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:36.933409 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:12:36.933591 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:36.933594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:36.933657 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:36.933759 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:36.934387 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.935065 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.936126 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:36.936378 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:36.936479 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:36.940348 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:45431: BlockingRecv error: recv got EOF from 127.9.133.129:45431 (error 108)
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":175,"threads_started":1}
Aug 07 06:12:37 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:37 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:37.174717 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:37.168180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:46829)
0807 06:12:37.168542 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:37.168556 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:37.168726 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:37.169092 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:37.169095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:37.169368 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:37.169509 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:37.169515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.170332 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.170337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:37.170970 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:37.170979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.171506 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.171509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:37.171606 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:37.172266 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:37.172280 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:37.174555 (+ 2275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
W20260807 06:12:37.174897 22729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:37.168229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:46829 (local address 127.9.133.190:36533)
0807 06:12:37.168417 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:37.168422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:37.168644 (+ 222us) server_negotiation.cc:407] Connection header received
0807 06:12:37.168838 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:37.168841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:37.168905 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:37.169055 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:37.169635 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.170219 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.171136 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.171377 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.171495 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:37.174737 (+ 3242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:46829: BlockingRecv error: recv got EOF from 127.9.133.130:46829 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 06:12:37 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:37 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:37.391415 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:37.385335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:58627)
0807 06:12:37.385666 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:37.385683 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:37.385807 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:37.386093 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:37.386096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:37.386327 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:37.386448 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:37.386454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.387238 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.387241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:37.387852 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:37.387861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.388275 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.388278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:37.388344 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:37.388912 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:37.388925 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:37.391255 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":125,"threads_started":1}
W20260807 06:12:37.391567 22729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:37.385366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:58627 (local address 127.9.133.190:36533)
0807 06:12:37.385559 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:37.385565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:37.385766 (+ 201us) server_negotiation.cc:407] Connection header received
0807 06:12:37.385897 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:37.385899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:37.385956 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:37.386028 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:37.386558 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.387136 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.387956 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.388176 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.388254 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:37.391411 (+ 3157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:58627: BlockingRecv error: recv got EOF from 127.9.133.131:58627 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 06:12:37 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:37 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:37.954250 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:37.941543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:51579)
0807 06:12:37.942014 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:37.942034 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:37.942181 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:37.942463 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:37.942466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:37.942714 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:37.942846 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:37.942852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.943723 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.943726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:37.944431 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:37.944441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.944911 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.944914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:37.945012 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:37.951188 (+ 6176us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:37.951205 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:37.953962 (+ 2757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":127,"threads_started":1}
W20260807 06:12:37.954391 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:37.941607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:51579 (local address 127.9.133.190:36533)
0807 06:12:37.941997 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:37.942013 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:37.942129 (+ 116us) server_negotiation.cc:407] Connection header received
0807 06:12:37.942274 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:37.942276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:37.942333 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:37.942416 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:37.942955 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.943600 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.944552 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:37.944797 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:37.944884 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:37.954193 (+ 9309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:51579: BlockingRecv error: recv got EOF from 127.9.133.129:51579 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":151,"threads_started":1}
Aug 07 06:12:38 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:38 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:38.182435 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:38.175901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:47771)
0807 06:12:38.176327 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:38.176346 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:38.176518 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:38.176845 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:38.176848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:38.177151 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:38.177289 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:38.177296 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.178137 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.178142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:38.178741 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:38.178749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.179293 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.179297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:38.179384 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:38.180018 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:38.180030 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:38.182187 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":119,"threads_started":1}
W20260807 06:12:38.182543 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:38.176082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:47771 (local address 127.9.133.190:36533)
0807 06:12:38.176305 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:38.176310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:38.176460 (+ 150us) server_negotiation.cc:407] Connection header received
0807 06:12:38.176641 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:38.176645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:38.176711 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:38.176813 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:38.177430 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.178021 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.178869 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.179173 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.179287 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:38.182382 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:47771: BlockingRecv error: recv got EOF from 127.9.133.130:47771 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 06:12:38 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:38 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:38.398908 22736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:38.392482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:38807)
0807 06:12:38.392851 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:38.392866 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:38.392993 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:38.393264 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:38.393266 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:38.393519 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:38.393642 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:38.393648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.394421 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.394423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:38.395118 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:38.395129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.395560 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.395563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:38.395631 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:38.396284 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:38.396297 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:38.398659 (+ 2362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":143,"threads_started":1}
W20260807 06:12:38.399047 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:38.392671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:38807 (local address 127.9.133.190:36533)
0807 06:12:38.392843 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:38.392847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:38.392945 (+ 98us) server_negotiation.cc:407] Connection header received
0807 06:12:38.393082 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:38.393084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:38.393141 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:38.393217 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:38.393749 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.394318 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.395229 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.395455 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.395540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:38.398869 (+ 3329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:38807: BlockingRecv error: recv got EOF from 127.9.133.131:38807 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 06:12:38 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:38 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:38.962329 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:38.955387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:39023)
0807 06:12:38.955818 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:38.955841 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:38.956056 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:38.956417 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:38.956420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:38.956773 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:38.956934 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:38.956944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.957869 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.957879 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:38.958494 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:38.958503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.959114 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.959118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:38.959211 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:38.959854 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:38.959868 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:38.962170 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":159,"threads_started":1}
W20260807 06:12:38.962503 22738 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:38.955475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:39023 (local address 127.9.133.190:36533)
0807 06:12:38.955838 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:38.955844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:38.955948 (+ 104us) server_negotiation.cc:407] Connection header received
0807 06:12:38.956175 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:38.956178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:38.956268 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:38.956387 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:38.957087 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.957719 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.958625 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:38.958909 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:38.959105 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:38.962339 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:39023: BlockingRecv error: recv got EOF from 127.9.133.129:39023 (error 108)
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
Aug 07 06:12:39 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:39 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:39.190088 22739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:39.183466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:45287)
0807 06:12:39.183879 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:39.183894 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:39.184031 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:39.184301 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:39.184304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:39.184544 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:39.184677 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:39.184684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.185470 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.185472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:39.186064 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:39.186073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.186570 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.186573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:39.186641 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:39.187285 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:39.187302 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:39.189914 (+ 2612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
W20260807 06:12:39.190229 22738 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:39.183616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:45287 (local address 127.9.133.190:36533)
0807 06:12:39.183795 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:39.183799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:39.183989 (+ 190us) server_negotiation.cc:407] Connection header received
0807 06:12:39.184125 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:39.184127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:39.184184 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:39.184259 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:39.184789 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.185368 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.186207 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.186456 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.186552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:39.190088 (+ 3536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:45287: BlockingRecv error: recv got EOF from 127.9.133.130:45287 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 06:12:39 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:39 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:39.407195 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:39.399936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:41639)
0807 06:12:39.400403 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:39.400428 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:39.400610 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:39.401029 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:39.401032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:39.401314 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:39.401451 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:39.401457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.402314 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.402319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:39.403317 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:39.403331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.403979 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.403982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:39.404073 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:39.404681 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:39.404694 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:39.406980 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":154,"threads_started":1}
W20260807 06:12:39.407464 22738 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:39.400106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:41639 (local address 127.9.133.190:36533)
0807 06:12:39.400350 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:39.400356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:39.400533 (+ 177us) server_negotiation.cc:407] Connection header received
0807 06:12:39.400741 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:39.400751 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:39.400814 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:39.401000 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:39.401587 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.402191 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.403469 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.403837 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.403963 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:39.407198 (+ 3235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:41639: BlockingRecv error: recv got EOF from 127.9.133.131:41639 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 07 06:12:39 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:39 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:39.970367 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:39.963481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:53837)
0807 06:12:39.963834 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:39.963849 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:39.964012 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:39.964345 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:39.964348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:39.964583 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:39.964701 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:39.964707 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.965795 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.965799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:39.966677 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:39.966690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.967215 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.967218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:39.967286 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:39.967887 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:39.967900 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:39.970159 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":124,"threads_started":1}
W20260807 06:12:39.970451 22742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:39.963549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:53837 (local address 127.9.133.190:36533)
0807 06:12:39.963841 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:39.963846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:39.963953 (+ 107us) server_negotiation.cc:407] Connection header received
0807 06:12:39.964146 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:39.964149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:39.964208 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:39.964303 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:39.964813 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.965679 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.966818 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:39.967113 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:39.967196 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:39.970294 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:53837: BlockingRecv error: recv got EOF from 127.9.133.129:53837 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
Aug 07 06:12:40 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:40 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:40.197451 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:40.191125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:48845)
0807 06:12:40.191506 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:40.191522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:40.191652 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:40.191928 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:40.191930 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:40.192173 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:40.192300 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:40.192306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.193119 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.193122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:40.193728 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:40.193737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.194212 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.194215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:40.194285 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:40.194949 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:40.194962 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:40.197293 (+ 2331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
W20260807 06:12:40.197676 22742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:40.191249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:48845 (local address 127.9.133.190:36533)
0807 06:12:40.191414 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:40.191419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:40.191605 (+ 186us) server_negotiation.cc:407] Connection header received
0807 06:12:40.191744 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:40.191747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:40.191805 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:40.191876 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:40.192408 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.192992 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.193842 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.194101 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.194186 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:40.197474 (+ 3288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:48845: BlockingRecv error: recv got EOF from 127.9.133.130:48845 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 06:12:40 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:40 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:40.414623 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:40.408131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:59133)
0807 06:12:40.408472 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:40.408488 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:40.408617 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:40.408880 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:40.408882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:40.409140 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:40.409266 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:40.409272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.410056 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.410059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:40.410726 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:40.410736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.411217 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.411221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:40.411317 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:40.411991 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:40.412005 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:40.414409 (+ 2404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":134,"threads_started":1}
W20260807 06:12:40.414695 22742 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:40.408214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:59133 (local address 127.9.133.190:36533)
0807 06:12:40.408406 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:40.408412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:40.408569 (+ 157us) server_negotiation.cc:407] Connection header received
0807 06:12:40.408705 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:40.408707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:40.408766 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:40.408840 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:40.409370 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.409943 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.410838 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.411093 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.411176 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:40.414564 (+ 3388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:59133: BlockingRecv error: recv got EOF from 127.9.133.131:59133 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 06:12:40 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:40 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:40.978189 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:40.971575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:48293)
0807 06:12:40.972036 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:40.972055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:40.972237 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:40.972617 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:40.972621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:40.972934 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:40.973120 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:40.973129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.974069 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.974072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:40.974679 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:40.974687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.975183 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.975185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:40.975258 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:40.975862 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:40.975874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:40.978018 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":149,"threads_started":1}
W20260807 06:12:40.978334 22746 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:40.971665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:48293 (local address 127.9.133.190:36533)
0807 06:12:40.972022 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:40.972027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:40.972167 (+ 140us) server_negotiation.cc:407] Connection header received
0807 06:12:40.972373 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:40.972377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:40.972441 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:40.972570 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:40.973282 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.973952 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.974824 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:40.975081 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:40.975160 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:40.978176 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:48293: BlockingRecv error: recv got EOF from 127.9.133.129:48293 (error 108)
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":130,"threads_started":1}
Aug 07 06:12:41 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:41 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:41.205695 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:41.198550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:46069)
0807 06:12:41.198899 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:41.198918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:41.199113 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:41.199479 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:41.199482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:41.199809 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:41.200038 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:41.200051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.200904 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.200911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:41.201755 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:41.201790 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.202381 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.202384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:41.202490 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:41.203217 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:41.203230 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:41.205489 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":107,"threads_started":1}
W20260807 06:12:41.205744 22746 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:41.198730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:46069 (local address 127.9.133.190:36533)
0807 06:12:41.198900 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:41.198906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:41.199046 (+ 140us) server_negotiation.cc:407] Connection header received
0807 06:12:41.199249 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:41.199255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:41.199334 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:41.199445 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:41.200198 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.200798 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.201897 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.202235 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.202359 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:41.205628 (+ 3269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:46069: BlockingRecv error: recv got EOF from 127.9.133.130:46069 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 06:12:41 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:41 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:41.422789 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:41.415964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:36651)
0807 06:12:41.416345 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:41.416361 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:41.416497 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:41.416791 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:41.416793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:41.417051 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:41.417182 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:41.417188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.418009 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.418012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:41.418640 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:41.418649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.419176 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.419179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:41.419259 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:41.419886 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:41.419900 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:41.422579 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":140,"threads_started":1}
W20260807 06:12:41.422859 22746 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:41.415828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:36651 (local address 127.9.133.190:36533)
0807 06:12:41.416088 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:41.416094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:41.416475 (+ 381us) server_negotiation.cc:407] Connection header received
0807 06:12:41.416600 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:41.416603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:41.416667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:41.416744 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:41.417292 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.417896 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.418758 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.419035 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.419158 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:41.422732 (+ 3574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:36651: BlockingRecv error: recv got EOF from 127.9.133.131:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 07 06:12:41 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:41 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:41.985710 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:41.979271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:51809)
0807 06:12:41.979705 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:41.979720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:41.979852 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:41.980147 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:41.980150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:41.980378 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:41.980514 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:41.980521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.981493 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.981496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:41.982080 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:41.982088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.982568 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.982570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:41.982640 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:41.983317 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:41.983330 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:41.985484 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":120,"threads_started":1}
W20260807 06:12:41.985791 22753 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:41.979328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:51809 (local address 127.9.133.190:36533)
0807 06:12:41.979666 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:41.979670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:41.979800 (+ 130us) server_negotiation.cc:407] Connection header received
0807 06:12:41.979940 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:41.979943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:41.979998 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:41.980105 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:41.980710 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.981365 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.982202 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:41.982453 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:41.982551 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:41.985631 (+ 3080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:51809: BlockingRecv error: recv got EOF from 127.9.133.129:51809 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
Aug 07 06:12:42 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:42 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:42.213194 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:42.206671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:43365)
0807 06:12:42.207179 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:42.207195 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:42.207326 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:42.207604 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:42.207607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:42.207903 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:42.208029 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:42.208036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.208852 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.208855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:42.209491 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:42.209534 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.210036 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.210039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:42.210115 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:42.210751 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:42.210764 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:42.213039 (+ 2275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":151,"threads_started":1}
W20260807 06:12:42.213336 22753 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:42.206762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:43365 (local address 127.9.133.190:36533)
0807 06:12:42.207075 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:42.207080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:42.207282 (+ 202us) server_negotiation.cc:407] Connection header received
0807 06:12:42.207421 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:42.207424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:42.207479 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:42.207556 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:42.208142 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.208729 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.209665 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.209909 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.209995 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:42.213173 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:43365: BlockingRecv error: recv got EOF from 127.9.133.130:43365 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 07 06:12:42 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:42 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:42.430220 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:42.423764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:32779)
0807 06:12:42.424214 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:42.424229 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:42.424390 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:42.424710 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:42.424714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:42.425014 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:42.425137 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:42.425143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.425980 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.425984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:42.426583 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:42.426591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.427155 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.427159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:42.427263 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:42.427915 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:42.427927 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:42.430064 (+ 2137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":133,"threads_started":1}
W20260807 06:12:42.430401 22753 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:42.423856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:32779 (local address 127.9.133.190:36533)
0807 06:12:42.424058 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:42.424066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:42.424314 (+ 248us) server_negotiation.cc:407] Connection header received
0807 06:12:42.424513 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:42.424517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:42.424584 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:42.424681 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:42.425263 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.425856 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.426715 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.426959 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.427148 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:42.430252 (+ 3104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:32779: BlockingRecv error: recv got EOF from 127.9.133.131:32779 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 06:12:42 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:42 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:42.996802 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:42.986843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:41535)
0807 06:12:42.987330 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:42.987346 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:42.987476 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:42.987770 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:42.987773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:42.988042 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:42.988170 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:42.988178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.989052 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.989055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:42.989643 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:42.989652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.990165 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.990169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:42.990263 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:42.994140 (+ 3877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:42.994155 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:42.996648 (+ 2493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":105,"threads_started":1}
W20260807 06:12:42.996989 22758 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:42.986898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:41535 (local address 127.9.133.190:36533)
0807 06:12:42.987278 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:42.987286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:42.987434 (+ 148us) server_negotiation.cc:407] Connection header received
0807 06:12:42.987575 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:42.987578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:42.987638 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:42.987734 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:42.988286 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.988948 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.989759 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:42.990030 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:42.990124 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:42.996829 (+ 6705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:41535: BlockingRecv error: recv got EOF from 127.9.133.129:41535 (error 108)
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":124,"threads_started":1}
Aug 07 06:12:43 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:43 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:43.221068 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:43.214111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:49353)
0807 06:12:43.214602 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:43.214618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:43.214746 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:43.215124 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:43.215126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:43.215378 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:43.215504 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:43.215511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.216363 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.216367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:43.217031 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:43.217041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.217477 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.217479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:43.217541 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:43.218139 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:43.218151 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:43.220913 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":226,"threads_started":1}
W20260807 06:12:43.221199 22758 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:43.214222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:49353 (local address 127.9.133.190:36533)
0807 06:12:43.214393 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:43.214397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:43.214699 (+ 302us) server_negotiation.cc:407] Connection header received
0807 06:12:43.214838 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:43.214841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:43.214896 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:43.214973 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:43.215640 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.216237 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.217146 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.217376 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.217459 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:43.221047 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:49353: BlockingRecv error: recv got EOF from 127.9.133.130:49353 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 06:12:43 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:43 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:43.437700 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:43.431206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:37607)
0807 06:12:43.431638 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:43.431653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:43.431822 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:43.432143 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:43.432145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:43.432420 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:43.432537 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:43.432543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.433398 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.433403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:43.433999 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:43.434009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.434522 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.434525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:43.434624 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:43.435254 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:43.435267 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:43.437471 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":146,"threads_started":1}
W20260807 06:12:43.437754 22758 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:43.431276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:37607 (local address 127.9.133.190:36533)
0807 06:12:43.431443 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:43.431449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:43.431745 (+ 296us) server_negotiation.cc:407] Connection header received
0807 06:12:43.431939 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:43.431944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:43.432013 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:43.432108 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:43.432677 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.433270 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.434135 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.434394 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:43.434504 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:43.437612 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:37607: BlockingRecv error: recv got EOF from 127.9.133.131:37607 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 06:12:44 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:44 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:44.004896 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:43.997836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:34811)
0807 06:12:43.998225 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:43.998241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:43.998362 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:43.998666 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:43.998668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:43.998943 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:43.999105 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:43.999113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.000028 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.000031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:44.000645 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:44.000656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.001189 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.001192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:44.001268 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:44.001950 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:44.001963 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:44.004679 (+ 2716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":146,"threads_started":1}
W20260807 06:12:44.004985 22767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:43.998038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:34811 (local address 127.9.133.190:36533)
0807 06:12:43.998301 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:43.998304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:43.998319 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:12:43.998464 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:43.998466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:43.998531 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:43.998616 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:43.999252 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:43.999906 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.000771 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.001065 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.001169 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:44.004830 (+ 3661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:34811: BlockingRecv error: recv got EOF from 127.9.133.129:34811 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
Aug 07 06:12:44 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:44 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:44.228752 22769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:44.222098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:43901)
0807 06:12:44.222461 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:44.222476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:44.222611 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:44.222884 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:44.222887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:44.223313 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:44.223464 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:44.223471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.224432 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.224435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:44.225030 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:44.225038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.225463 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.225465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:44.225542 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:44.226162 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:44.226175 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:44.228593 (+ 2418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":126,"threads_started":1}
W20260807 06:12:44.229210 22767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:44.222191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:43901 (local address 127.9.133.190:36533)
0807 06:12:44.222373 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:44.222379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:44.222563 (+ 184us) server_negotiation.cc:407] Connection header received
0807 06:12:44.222704 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:44.222707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:44.222766 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:44.222840 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:44.223586 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.224330 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.225136 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.225361 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.225440 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:44.228833 (+ 3393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:43901: BlockingRecv error: recv got EOF from 127.9.133.130:43901 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 06:12:44 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:44 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:44.445335 22770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:44.438927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:35103)
0807 06:12:44.439340 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:44.439357 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:44.439495 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:44.439822 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:44.439829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:44.440114 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:44.440275 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:44.440284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.441365 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.441368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:44.441999 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:44.442008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.442422 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.442425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:44.442490 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:44.443182 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:44.443195 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:44.445192 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":160,"threads_started":1}
W20260807 06:12:44.445482 22767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:44.438972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:35103 (local address 127.9.133.190:36533)
0807 06:12:44.439183 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:44.439188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:44.439455 (+ 267us) server_negotiation.cc:407] Connection header received
0807 06:12:44.439587 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:44.439592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:44.439667 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:44.439749 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:44.440413 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.441262 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.442103 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:44.442316 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:44.442394 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:44.445343 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:35103: BlockingRecv error: recv got EOF from 127.9.133.131:35103 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 06:12:45 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:45 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:45.013062 22771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:45.006089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:55951)
0807 06:12:45.006620 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:45.006635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:45.006759 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:45.007101 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:45.007104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:45.007348 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:45.007492 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:45.007499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.008395 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.008398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:45.009114 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:45.009127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.009645 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.009649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:45.009734 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:45.010470 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:45.010485 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:45.012869 (+ 2384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":145,"threads_started":1}
W20260807 06:12:45.013273 22772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:45.006179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:55951 (local address 127.9.133.190:36533)
0807 06:12:45.006503 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:45.006509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:45.006715 (+ 206us) server_negotiation.cc:407] Connection header received
0807 06:12:45.006849 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:45.006851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:45.006909 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:45.007045 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:45.007613 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.008235 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.009251 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.009509 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.009603 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:45.013092 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:55951: BlockingRecv error: recv got EOF from 127.9.133.129:55951 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":94,"threads_started":1}
Aug 07 06:12:45 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:45 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:45.236690 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:45.229850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:55487)
0807 06:12:45.230205 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:45.230221 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:45.230349 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:45.230634 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:45.230636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:45.230936 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:45.231120 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:45.231129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.231980 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.231983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:45.232607 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:45.232618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.233190 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.233194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:45.233411 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:45.234157 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:45.234170 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:45.236505 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":145,"threads_started":1}
W20260807 06:12:45.236770 22772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:45.229954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:55487 (local address 127.9.133.190:36533)
0807 06:12:45.230158 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:45.230162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:45.230300 (+ 138us) server_negotiation.cc:407] Connection header received
0807 06:12:45.230439 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:45.230441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:45.230497 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:45.230593 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:45.231242 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.231848 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.232739 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.233031 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.233169 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:45.236654 (+ 3485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:55487: BlockingRecv error: recv got EOF from 127.9.133.130:55487 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 06:12:45 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:45 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:45.453137 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:45.446581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:39387)
0807 06:12:45.446949 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:45.446964 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:45.447162 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:45.447464 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:45.447467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:45.447740 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:45.447860 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:45.447866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.448813 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.448819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:45.449426 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:45.449435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.449912 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.449916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:45.450024 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:45.450611 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:45.450623 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:45.452871 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
W20260807 06:12:45.453125 22772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:45.446653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:39387 (local address 127.9.133.190:36533)
0807 06:12:45.446850 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:45.446855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:45.447080 (+ 225us) server_negotiation.cc:407] Connection header received
0807 06:12:45.447277 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:45.447281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:45.447344 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:45.447431 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:45.447980 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.448701 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.449559 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:45.449793 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:45.449905 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:45.453011 (+ 3106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:39387: BlockingRecv error: recv got EOF from 127.9.133.131:39387 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 06:12:46 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:46 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:46.021494 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:46.014303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:41937)
0807 06:12:46.014738 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:46.014754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:46.014883 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:46.015304 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:46.015307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:46.015550 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:46.015684 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:46.015691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.016537 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.016541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:46.017280 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:46.017293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.017848 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.017852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:46.017964 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:46.018630 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:46.018644 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:46.021331 (+ 2687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":156,"threads_started":1}
W20260807 06:12:46.021771 22776 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:46.014332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:41937 (local address 127.9.133.190:36533)
0807 06:12:46.014730 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:46.014734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:46.014833 (+ 99us) server_negotiation.cc:407] Connection header received
0807 06:12:46.015051 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:46.015056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:46.015134 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:46.015246 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:46.015800 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.016397 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.017448 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.017707 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.017796 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:46.021589 (+ 3793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:41937: BlockingRecv error: recv got EOF from 127.9.133.129:41937 (error 108)
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":143,"threads_started":1}
Aug 07 06:12:46 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:46 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:46.244782 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:46.237777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:34409)
0807 06:12:46.238152 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:46.238167 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:46.238339 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:46.238679 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:46.238682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:46.239133 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:46.239266 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:46.239275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.240225 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.240228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:46.240829 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:46.240838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.241324 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.241326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:46.241394 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:46.242050 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:46.242062 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:46.244573 (+ 2511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":154,"threads_started":1}
W20260807 06:12:46.244953 22776 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:46.237893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34409 (local address 127.9.133.190:36533)
0807 06:12:46.238087 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:46.238091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:46.238270 (+ 179us) server_negotiation.cc:407] Connection header received
0807 06:12:46.238476 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:46.238479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:46.238546 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:46.238632 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:46.239416 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.240098 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.240959 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.241216 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.241323 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:46.244749 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:34409: BlockingRecv error: recv got EOF from 127.9.133.130:34409 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 06:12:46 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:46 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:46.461727 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:46.454403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:49247)
0807 06:12:46.454784 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:46.454801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:46.454920 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:46.455221 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:46.455223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:46.455456 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:46.455626 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:46.455636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.456508 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.456512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:46.457136 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:46.457146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.457894 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.457897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:46.457974 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:46.458727 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:46.458741 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:46.461554 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":154,"threads_started":1}
W20260807 06:12:46.461907 22776 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:46.454565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:49247 (local address 127.9.133.190:36533)
0807 06:12:46.454750 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:46.454754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:46.454911 (+ 157us) server_negotiation.cc:407] Connection header received
0807 06:12:46.455029 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:46.455032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:46.455090 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:46.455171 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:46.455749 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.456402 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.457420 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:46.457762 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:46.457867 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:46.461719 (+ 3852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:49247: BlockingRecv error: recv got EOF from 127.9.133.131:49247 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 06:12:47 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:47 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:47.033668 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:47.022622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:42609)
0807 06:12:47.023195 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:47.023211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:47.023343 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:47.023683 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:47.023687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:47.023988 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:47.024130 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:47.024137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.025029 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.025032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:47.025640 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:47.025650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.026157 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.026160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:47.026224 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:47.030551 (+ 4327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:47.030567 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:47.033435 (+ 2868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":121,"threads_started":1}
W20260807 06:12:47.033974 22783 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:47.022681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:42609 (local address 127.9.133.190:36533)
0807 06:12:47.022995 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:47.023027 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:47.023368 (+ 341us) server_negotiation.cc:407] Connection header received
0807 06:12:47.023432 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:47.023435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:47.023494 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:47.023596 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:47.024261 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.024884 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.025764 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.026032 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.026139 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:47.033648 (+ 7509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:42609: BlockingRecv error: recv got EOF from 127.9.133.129:42609 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":82,"threads_started":1}
Aug 07 06:12:47 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:47 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:47.252892 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:47.245940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:35255)
0807 06:12:47.246293 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:47.246318 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:47.246491 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:47.246826 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:47.246830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:47.247150 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:47.247280 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:47.247287 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.248158 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.248164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:47.248779 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:47.248788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.249315 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.249319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:47.249417 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:47.250126 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:47.250140 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:47.252658 (+ 2518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":118,"threads_started":1}
W20260807 06:12:47.253115 22783 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:47.246010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:35255 (local address 127.9.133.190:36533)
0807 06:12:47.246217 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:47.246222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:47.246414 (+ 192us) server_negotiation.cc:407] Connection header received
0807 06:12:47.246643 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:47.246646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:47.246701 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:47.246792 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:47.247414 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.248037 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.248916 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.249176 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.249295 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:47.252869 (+ 3574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:35255: BlockingRecv error: recv got EOF from 127.9.133.130:35255 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 06:12:47 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:47 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:47.469451 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:47.462777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:60581)
0807 06:12:47.463194 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:47.463208 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:47.463372 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:47.463686 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:47.463689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:47.463969 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:47.464116 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:47.464123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.464976 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.464982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:47.465594 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:47.465603 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.466109 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.466112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:47.466208 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:47.466854 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:47.466866 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:47.469230 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":153,"threads_started":1}
W20260807 06:12:47.469579 22783 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:47.462833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:60581 (local address 127.9.133.190:36533)
0807 06:12:47.463048 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:47.463053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:47.463296 (+ 243us) server_negotiation.cc:407] Connection header received
0807 06:12:47.463485 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:47.463488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:47.463561 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:47.463656 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:47.464246 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.464857 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.465724 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:47.465982 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:47.466100 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:47.469411 (+ 3311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:60581: BlockingRecv error: recv got EOF from 127.9.133.131:60581 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 06:12:48 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:48 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:48.047253 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:48.034725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:48233)
0807 06:12:48.035226 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:48.035243 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:48.035397 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:48.035700 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:48.035703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:48.035953 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:48.036112 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:48.036119 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.036952 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.036955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:48.037783 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:48.037795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.038369 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.038373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:48.038464 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:48.044642 (+ 6178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:48.044656 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:48.047029 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":183,"threads_started":1}
W20260807 06:12:48.047353 22790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:48.034816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:48233 (local address 127.9.133.190:36533)
0807 06:12:48.035226 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:48.035230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:48.035362 (+ 132us) server_negotiation.cc:407] Connection header received
0807 06:12:48.035498 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:48.035500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:48.035558 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:48.035652 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:48.036233 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.036842 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.037906 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.038217 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.038376 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:48.047202 (+ 8826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:48233: BlockingRecv error: recv got EOF from 127.9.133.129:48233 (error 108)
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":116,"threads_started":1}
Aug 07 06:12:48 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:48 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:48.261212 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:48.254087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:56275)
0807 06:12:48.254476 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:48.254491 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:48.254664 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:48.255213 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:48.255217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:48.255531 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:48.255693 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:48.255702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.256610 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.256613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:48.257346 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:48.257359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.257818 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.257821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:48.257960 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:48.258672 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:48.258686 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:48.261056 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":110,"threads_started":1}
W20260807 06:12:48.261343 22790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:48.254180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:56275 (local address 127.9.133.190:36533)
0807 06:12:48.254413 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:48.254419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:48.254582 (+ 163us) server_negotiation.cc:407] Connection header received
0807 06:12:48.254864 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:48.254871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:48.254948 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:48.255087 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:48.255857 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.256467 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.257463 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.257714 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.257799 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:48.261203 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:56275: BlockingRecv error: recv got EOF from 127.9.133.130:56275 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 06:12:48 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:48 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:48.476770 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:48.470455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:40623)
0807 06:12:48.470809 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:48.470824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:48.470957 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:48.471300 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:48.471302 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:48.471558 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:48.471692 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:48.471697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.472526 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.472530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:48.473171 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:48.473182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.473615 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.473618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:48.473689 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:48.474297 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:48.474309 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:48.476557 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
W20260807 06:12:48.476809 22790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:48.470589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:40623 (local address 127.9.133.190:36533)
0807 06:12:48.470800 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:48.470805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:48.470908 (+ 103us) server_negotiation.cc:407] Connection header received
0807 06:12:48.471098 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:48.471101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:48.471159 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:48.471230 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:48.471797 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.472400 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.473288 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:48.473513 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:48.473593 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:48.476688 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:40623: BlockingRecv error: recv got EOF from 127.9.133.131:40623 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 06:12:49 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:49 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:49.055490 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:49.048307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:33379)
0807 06:12:49.048665 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:49.048682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:49.048802 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:49.049129 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:49.049131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:49.049367 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:49.049495 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:49.049501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.050552 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.050556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:49.051232 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:49.051243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.051841 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.051845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:49.051972 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:49.052689 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:49.052704 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:49.055257 (+ 2553us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
W20260807 06:12:49.055729 22796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:49.048485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:33379 (local address 127.9.133.190:36533)
0807 06:12:49.048820 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:49.048825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:49.048851 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:12:49.048914 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:49.048916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:49.048975 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:49.049094 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:49.049662 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.050412 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.051388 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.051678 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.051786 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:49.055566 (+ 3780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:33379: BlockingRecv error: recv got EOF from 127.9.133.129:33379 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":128,"threads_started":1}
Aug 07 06:12:49 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:49 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:49.269562 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:49.262465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:37673)
0807 06:12:49.262947 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:49.262966 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:49.263169 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:49.263634 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:49.263639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:49.263914 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:49.264101 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:49.264114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.265145 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.265148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:49.265793 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:49.265802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.266315 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.266319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:49.266410 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:49.267101 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:49.267115 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:49.269335 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":165,"threads_started":1}
W20260807 06:12:49.269644 22796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:49.262641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:37673 (local address 127.9.133.190:36533)
0807 06:12:49.262828 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:49.262834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:49.263127 (+ 293us) server_negotiation.cc:407] Connection header received
0807 06:12:49.263279 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:49.263283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:49.263389 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:49.263551 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:49.264246 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.265025 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.265929 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.266169 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.266258 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:49.269505 (+ 3247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:37673: BlockingRecv error: recv got EOF from 127.9.133.130:37673 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 06:12:49 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:49 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:49.484532 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:49.477746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:38809)
0807 06:12:49.478143 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:49.478158 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:49.478335 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:49.478655 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:49.478658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:49.478944 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:49.479093 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:49.479100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.479918 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.479924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:49.480602 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:49.480611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.481174 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.481178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:49.481275 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:49.482036 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:49.482047 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:49.484361 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":124,"threads_started":1}
W20260807 06:12:49.484618 22796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:49.477846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:38809 (local address 127.9.133.190:36533)
0807 06:12:49.478066 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:49.478072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:49.478253 (+ 181us) server_negotiation.cc:407] Connection header received
0807 06:12:49.478462 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:49.478465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:49.478531 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:49.478628 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:49.479216 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.479796 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.480720 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:49.481047 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:49.481158 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:49.484497 (+ 3339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:38809: BlockingRecv error: recv got EOF from 127.9.133.131:38809 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 06:12:50 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:50 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:50.064312 22803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:50.056755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:56195)
0807 06:12:50.057290 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:50.057309 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:50.057478 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:50.057827 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:50.057832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:50.058135 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:50.058308 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:50.058318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.059433 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.059436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:50.060071 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:50.060082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.060613 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.060617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:50.060709 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:50.061339 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:50.061353 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:50.064089 (+ 2736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":155,"threads_started":1}
W20260807 06:12:50.064436 22804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:50.056919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:56195 (local address 127.9.133.190:36533)
0807 06:12:50.057328 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:50.057334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:50.057403 (+ 69us) server_negotiation.cc:407] Connection header received
0807 06:12:50.057610 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:50.057612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:50.057668 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:50.057760 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:50.058505 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.059309 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.060207 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.060476 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.060600 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:50.064274 (+ 3674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:56195: BlockingRecv error: recv got EOF from 127.9.133.129:56195 (error 108)
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":106,"threads_started":1}
Aug 07 06:12:50 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:50 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:50.277705 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:50.270761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:52391)
0807 06:12:50.271188 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:50.271202 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:50.271376 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:50.271759 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:50.271762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:50.272058 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:50.272243 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:50.272251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.273140 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.273143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:50.273773 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:50.273790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.274394 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.274398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:50.274493 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:50.275205 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:50.275219 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:50.277536 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":117,"threads_started":1}
W20260807 06:12:50.277925 22804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:50.270877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:52391 (local address 127.9.133.190:36533)
0807 06:12:50.271120 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:50.271125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:50.271329 (+ 204us) server_negotiation.cc:407] Connection header received
0807 06:12:50.271501 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:50.271523 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:50.271600 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:50.271694 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:50.272374 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.273017 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.273939 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.274233 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.274362 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:50.277720 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:52391: BlockingRecv error: recv got EOF from 127.9.133.130:52391 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 06:12:50 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:50 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.131@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:50.493574 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:50.486058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.131:44155)
0807 06:12:50.486440 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:50.486457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:50.486608 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:50.487092 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:50.487096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:50.487437 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:50.487605 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:50.487614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.488783 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.488786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:50.489427 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:50.489436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.490066 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.490069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:50.490146 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:50.490885 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:50.490899 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:50.493367 (+ 2468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":135,"threads_started":1}
W20260807 06:12:50.493784 22804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:50.486132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:44155 (local address 127.9.133.190:36533)
0807 06:12:50.486321 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:50.486327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:50.486560 (+ 233us) server_negotiation.cc:407] Connection header received
0807 06:12:50.486727 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:50.486731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:50.486802 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:50.486899 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:50.487799 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.488663 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.489556 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:50.489933 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:50.490036 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:50.493627 (+ 3591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.131:44155: BlockingRecv error: recv got EOF from 127.9.133.131:44155 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 06:12:51 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:51 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.129@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:51.072976 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:51.065668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.129:43271)
0807 06:12:51.066118 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:51.066135 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:51.066259 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:51.066599 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:51.066601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:51.066843 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:51.066971 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:51.066978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.068084 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.068088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:51.068761 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:51.068771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.069354 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.069358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:51.069451 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:51.070171 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:51.070186 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:51.072738 (+ 2552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":160,"threads_started":1}
W20260807 06:12:51.073227 22809 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:51.065846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:43271 (local address 127.9.133.190:36533)
0807 06:12:51.066180 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:51.066185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:51.066204 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:12:51.066376 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:51.066379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:51.066442 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:51.066543 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:51.067154 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.067858 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.068950 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.069217 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.069314 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:51.073050 (+ 3736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.129:43271: BlockingRecv error: recv got EOF from 127.9.133.129:43271 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":90,"threads_started":1}
Aug 07 06:12:51 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 06:12:51 dist-test-slave-h3ft krb5kdc[21246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.133.130@KRBTEST.COM for oryx/127.9.133.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 06:12:51.285789 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:51.278874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36533 (local address 127.9.133.130:52289)
0807 06:12:51.279304 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:51.279324 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:51.279481 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:51.279824 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:51.279827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:51.280094 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:51.280213 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:51.280244 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.281110 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.281115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:51.281718 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:51.281727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.282470 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.282472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:51.282601 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:51.283397 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:51.283414 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:51.285641 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.133.190:36533: Server oryx/127.9.133.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":143,"threads_started":1}
W20260807 06:12:51.286001 22809 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:51.279060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:52289 (local address 127.9.133.190:36533)
0807 06:12:51.279261 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:51.279267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:51.279408 (+ 141us) server_negotiation.cc:407] Connection header received
0807 06:12:51.279611 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:51.279616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:51.279685 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:51.279791 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:51.280392 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.280982 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.281878 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.282149 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.282281 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:51.285809 (+ 3528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.133.130:52289: BlockingRecv error: recv got EOF from 127.9.133.130:52289 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 06:12:51.366257 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 22089
I20260807 06:12:51.382309 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 22313
I20260807 06:12:51.401906 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 22513
I20260807 06:12:51.419832 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 21990
2026-08-07T06:12:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18871 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T06:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:51Z Disabled control of system clock
I20260807 06:12:51.464730 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35469
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:45067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:35469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 06:12:51.600791 22816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:51.601161 22816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:51.601200 22816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:51.605572 22816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:51.605660 22816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:51.605676 22816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:51.605695 22816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:51.605711 22816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:51.610925 22816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:35469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:51.612526 22816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:51.614401 22816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:51.624729 22827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:51.624729 22824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:51.624782 22825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:51.625115 22816 server_base.cc:1034] running on GCE node
I20260807 06:12:51.625967 22816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:51.627526 22816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:51.628738 22816 hybrid_clock.cc:648] HybridClock initialized: now 1786083171628691 us; error 65 us; skew 500 ppm
I20260807 06:12:51.631461 22816 webserver.cc:460] Webserver started at http://127.9.133.190:42243/ using document root <none> and password file <none>
I20260807 06:12:51.632289 22816 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:51.632395 22816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:51.632704 22816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:51.635375 22816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "2c406d9e2ed14bb59422b7adfe777173"
format_stamp: "Formatted at 2026-08-07 06:12:51 on dist-test-slave-h3ft"
server_key: "6ce0288e3718272a9d1824847561601a"
server_key_iv: "32cd2e24c17c8c5e8eb0b891e834ea5b"
server_key_version: "encryptionkey@0"
I20260807 06:12:51.636061 22816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "2c406d9e2ed14bb59422b7adfe777173"
format_stamp: "Formatted at 2026-08-07 06:12:51 on dist-test-slave-h3ft"
server_key: "6ce0288e3718272a9d1824847561601a"
server_key_iv: "32cd2e24c17c8c5e8eb0b891e834ea5b"
server_key_version: "encryptionkey@0"
I20260807 06:12:51.640673 22816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:51.644044 22842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:51.645665 22816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:12:51.645840 22816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "2c406d9e2ed14bb59422b7adfe777173"
format_stamp: "Formatted at 2026-08-07 06:12:51 on dist-test-slave-h3ft"
server_key: "6ce0288e3718272a9d1824847561601a"
server_key_iv: "32cd2e24c17c8c5e8eb0b891e834ea5b"
server_key_version: "encryptionkey@0"
I20260807 06:12:51.645998 22816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:51.665944 22816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:51.667091 22816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:51.667368 22816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:51.676417 22816 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:35469
I20260807 06:12:51.676455 22914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:35469 every 8 connection(s)
I20260807 06:12:51.677685 22816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:51.681757 22916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:51.683274 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 22816
I20260807 06:12:51.683434 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:51.683877 9750 external_mini_cluster.cc:1428] Setting key 46ca02a41d320d00b7320eae5f4b4a30
I20260807 06:12:51.689791 22916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173: Bootstrap starting.
I20260807 06:12:51.693053 22916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:51.694041 22916 log.cc:824] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:51.695798 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:51.685880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.9.133.190:35469)
0807 06:12:51.686732 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:51.686750 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:51.686796 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:12:51.687656 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:51.687681 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:51.688248 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:51.688671 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:51.689211 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.690413 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.691479 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:51.692023 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:51.692213 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:51.693429 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:51.693435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:51.693456 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:51.693976 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:51.694054 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:51.694079 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:51.694222 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:51.694394 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 06:12:51.694739 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":180,"threads_started":1}
I20260807 06:12:51.696746 22916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173: No bootstrap required, opened a new log
I20260807 06:12:51.701344 22916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c406d9e2ed14bb59422b7adfe777173" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35469 } }
I20260807 06:12:51.701699 22916 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:51.701793 22916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c406d9e2ed14bb59422b7adfe777173, State: Initialized, Role: FOLLOWER
I20260807 06:12:51.702345 22916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c406d9e2ed14bb59422b7adfe777173" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35469 } }
I20260807 06:12:51.702540 22916 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:51.702626 22916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:51.702775 22916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:51.704237 22916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c406d9e2ed14bb59422b7adfe777173" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35469 } }
I20260807 06:12:51.704697 22916 leader_election.cc:302] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c406d9e2ed14bb59422b7adfe777173; no voters:
I20260807 06:12:51.705113 22916 leader_election.cc:288] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:51.705263 22924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:51.705571 22924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [term 1 LEADER]: Becoming Leader. State: Replica: 2c406d9e2ed14bb59422b7adfe777173, State: Running, Role: LEADER
I20260807 06:12:51.705978 22924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c406d9e2ed14bb59422b7adfe777173" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35469 } }
I20260807 06:12:51.706724 22916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:51.708284 22929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c406d9e2ed14bb59422b7adfe777173. Latest consensus state: current_term: 1 leader_uuid: "2c406d9e2ed14bb59422b7adfe777173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c406d9e2ed14bb59422b7adfe777173" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35469 } } }
I20260807 06:12:51.708290 22928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c406d9e2ed14bb59422b7adfe777173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c406d9e2ed14bb59422b7adfe777173" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35469 } } }
I20260807 06:12:51.708455 22928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:51.708455 22929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:51.708968 22938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:51.713663 22938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:51.721352 22938 catalog_manager.cc:1347] Generated new cluster ID: 5a45f873f21c470a95eae59ffed808c6
I20260807 06:12:51.721474 22938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:51.734858 22938 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:51.736464 22938 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:51.743829 22938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c406d9e2ed14bb59422b7adfe777173: Generated new TSK 0
I20260807 06:12:51.745119 22938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:51.760226 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35469
--builtin_ntp_servers=127.9.133.148:45067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 06:12:51.915830 22956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:51.916199 22956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:51.916241 22956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:51.920890 22956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:51.920960 22956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:51.921056 22956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:51.926108 22956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:51.927670 22956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:51.929492 22956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:12:51.942180 22956 server_base.cc:1034] running on GCE node
W20260807 06:12:51.942118 22963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:51.942118 22966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:51.942135 22964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:51.943106 22956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:51.943945 22956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:51.945192 22956 hybrid_clock.cc:648] HybridClock initialized: now 1786083171945149 us; error 60 us; skew 500 ppm
I20260807 06:12:51.947772 22956 webserver.cc:460] Webserver started at http://127.9.133.129:44023/ using document root <none> and password file <none>
I20260807 06:12:51.948666 22956 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:51.948796 22956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:51.949123 22956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:51.952018 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "b6d9e664cfd24b1ebdb6f4b295af9586"
format_stamp: "Formatted at 2026-08-07 06:12:51 on dist-test-slave-h3ft"
server_key: "2e621b9efc4051cb8c8714b1a382df66"
server_key_iv: "ec8dcd1b095c37647a00c8c2a711451d"
server_key_version: "encryptionkey@0"
I20260807 06:12:51.952728 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "b6d9e664cfd24b1ebdb6f4b295af9586"
format_stamp: "Formatted at 2026-08-07 06:12:51 on dist-test-slave-h3ft"
server_key: "2e621b9efc4051cb8c8714b1a382df66"
server_key_iv: "ec8dcd1b095c37647a00c8c2a711451d"
server_key_version: "encryptionkey@0"
I20260807 06:12:51.957384 22956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:51.960803 22973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:51.962528 22956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:12:51.962708 22956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "b6d9e664cfd24b1ebdb6f4b295af9586"
format_stamp: "Formatted at 2026-08-07 06:12:51 on dist-test-slave-h3ft"
server_key: "2e621b9efc4051cb8c8714b1a382df66"
server_key_iv: "ec8dcd1b095c37647a00c8c2a711451d"
server_key_version: "encryptionkey@0"
I20260807 06:12:51.962872 22956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:51.977887 22956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:51.978930 22956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:51.979254 22956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:51.980070 22956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:51.981338 22956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:51.981416 22956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:51.981519 22956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:51.981570 22956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:51.992841 22956 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:32773
I20260807 06:12:51.992869 23147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:32773 every 8 connection(s)
I20260807 06:12:51.993981 22956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:52.002471 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 22956
I20260807 06:12:52.002619 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:52.003031 9750 external_mini_cluster.cc:1428] Setting key 044831b4d66a7be1a6ad3e9b89a8f54c
I20260807 06:12:52.005522 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35469
--builtin_ntp_servers=127.9.133.148:45067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 06:12:52.005905 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:51.996384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:35317 (local address 127.9.133.190:35469)
0807 06:12:51.996566 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:51.996571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:51.998009 (+ 1438us) server_negotiation.cc:407] Connection header received
0807 06:12:51.999720 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:51.999724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:51.999791 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:51.999879 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:52.000995 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.001648 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.003448 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.003786 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.003893 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:52.005324 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:52.005328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:52.005330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:52.005344 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:52.005369 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:52.005378 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:52.005458 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:52.005645 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 06:12:52.005732 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:12:52.007135 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:51.996944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35469 (local address 127.9.133.129:35317)
0807 06:12:51.997786 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:51.997835 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:51.999481 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:52.000117 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:52.000125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:52.000222 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:52.000813 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:52.000826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.001812 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.001816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:52.003317 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:52.003329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.003968 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.003973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:52.004186 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:52.005008 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:52.005011 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:52.005066 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:52.005148 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:52.005473 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:52.005484 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:52.005490 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:12:52.005645 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 06:12:52.005993 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":125,"threads_started":1}
I20260807 06:12:52.008642 23150 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35469
I20260807 06:12:52.008937 23150 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:52.009613 23150 heartbeater.cc:511] Master 127.9.133.190:35469 requested a full tablet report, sending...
I20260807 06:12:52.011615 22864 ts_manager.cc:194] Registered new tserver with Master: b6d9e664cfd24b1ebdb6f4b295af9586 (127.9.133.129:32773)
I20260807 06:12:52.013062 22864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.129:35317
W20260807 06:12:52.142761 23154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:52.143224 23154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:52.143319 23154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:52.147675 23154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:52.147786 23154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:52.147886 23154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:52.153462 23154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:52.155089 23154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:52.156932 23154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:52.168919 23168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:52.168862 23161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:52.168861 23165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:52.169253 23154 server_base.cc:1034] running on GCE node
I20260807 06:12:52.169839 23154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:52.170686 23154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:52.171872 23154 hybrid_clock.cc:648] HybridClock initialized: now 1786083172171855 us; error 31 us; skew 500 ppm
I20260807 06:12:52.174122 23154 webserver.cc:460] Webserver started at http://127.9.133.130:46341/ using document root <none> and password file <none>
I20260807 06:12:52.174809 23154 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:52.174877 23154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:52.175163 23154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:52.177610 23154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "943d6e8d35ad40bb8fd32571b5ec0752"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "01d902545dcbdb837827bbcabb3771bd"
server_key_iv: "ae96a5d18d84d666cff7819cd4dd24a9"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.178212 23154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "943d6e8d35ad40bb8fd32571b5ec0752"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "01d902545dcbdb837827bbcabb3771bd"
server_key_iv: "ae96a5d18d84d666cff7819cd4dd24a9"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.182580 23154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:12:52.185688 23180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.187310 23154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:12:52.187459 23154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "943d6e8d35ad40bb8fd32571b5ec0752"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "01d902545dcbdb837827bbcabb3771bd"
server_key_iv: "ae96a5d18d84d666cff7819cd4dd24a9"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.187621 23154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:52.212977 23154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:52.213896 23154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:52.214138 23154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:52.214880 23154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:52.216187 23154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:52.216266 23154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.216362 23154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:52.216406 23154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.227031 23154 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:37097
I20260807 06:12:52.227066 23360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:37097 every 8 connection(s)
I20260807 06:12:52.228217 23154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:52.236375 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 23154
I20260807 06:12:52.236543 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:52.236968 9750 external_mini_cluster.cc:1428] Setting key 2bf3287e77e1f1a9520d91e0911d5b97
I20260807 06:12:52.239781 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35469
--builtin_ntp_servers=127.9.133.148:45067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 06:12:52.239810 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:52.230462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:55307 (local address 127.9.133.190:35469)
0807 06:12:52.230627 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:52.230632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:52.232037 (+ 1405us) server_negotiation.cc:407] Connection header received
0807 06:12:52.233630 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:52.233634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:52.233711 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:52.233812 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:52.234944 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.235673 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.237085 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.237439 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.237581 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:52.239199 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:52.239204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:52.239208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:52.239225 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:52.239253 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:52.239265 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:52.239346 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:52.239526 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 06:12:52.239616 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:12:52.240844 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:52.230964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35469 (local address 127.9.133.130:55307)
0807 06:12:52.231841 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:52.231891 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:52.233368 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:52.234033 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:52.234041 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:52.234142 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:52.234759 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:52.234774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.235814 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.235817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:52.236940 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:52.236951 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.237673 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.237680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:52.237950 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:52.238815 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:52.238818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:52.238888 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:52.238972 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:52.239364 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:52.239372 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:52.239377 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:12:52.239484 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 06:12:52.239772 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":123,"threads_started":1}
I20260807 06:12:52.242211 23361 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35469
I20260807 06:12:52.242494 23361 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:52.243221 23361 heartbeater.cc:511] Master 127.9.133.190:35469 requested a full tablet report, sending...
I20260807 06:12:52.244477 22864 ts_manager.cc:194] Registered new tserver with Master: 943d6e8d35ad40bb8fd32571b5ec0752 (127.9.133.130:37097)
I20260807 06:12:52.245256 22864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.130:55307
W20260807 06:12:52.380904 23367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:52.381294 23367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:52.381376 23367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:52.385677 23367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:52.385771 23367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:52.385875 23367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:52.390993 23367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:52.392616 23367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:52.394410 23367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:52.406581 23381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:52.406563 23373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:52.406569 23376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:52.407162 23367 server_base.cc:1034] running on GCE node
I20260807 06:12:52.407797 23367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:52.408737 23367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:52.409977 23367 hybrid_clock.cc:648] HybridClock initialized: now 1786083172409962 us; error 41 us; skew 500 ppm
I20260807 06:12:52.412562 23367 webserver.cc:460] Webserver started at http://127.9.133.131:33167/ using document root <none> and password file <none>
I20260807 06:12:52.413352 23367 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:52.413470 23367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:52.413762 23367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:52.416544 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "9b6aeec465d34b2198f0f00512d2e90a"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "285c90fa8bf8f9f32da70eb4f95581ea"
server_key_iv: "3a3080b680b788750ceda0a15fe64613"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.417272 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "9b6aeec465d34b2198f0f00512d2e90a"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "285c90fa8bf8f9f32da70eb4f95581ea"
server_key_iv: "3a3080b680b788750ceda0a15fe64613"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.422336 23367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:12:52.425823 23389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.427778 23367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:52.427980 23367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "9b6aeec465d34b2198f0f00512d2e90a"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "285c90fa8bf8f9f32da70eb4f95581ea"
server_key_iv: "3a3080b680b788750ceda0a15fe64613"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.428155 23367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:52.452368 23367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:52.453435 23367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:52.453713 23367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:52.454550 23367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:52.455889 23367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:52.455987 23367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.456080 23367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:52.456131 23367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.467808 23367 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:42055
I20260807 06:12:52.467864 23562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:42055 every 8 connection(s)
I20260807 06:12:52.469064 23367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:52.470677 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 23367
I20260807 06:12:52.470834 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:52.471349 9750 external_mini_cluster.cc:1428] Setting key 0276bad0a1d2d3d9078d249ed37fabc0
I20260807 06:12:52.480893 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:52.471846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:48093 (local address 127.9.133.190:35469)
0807 06:12:52.472097 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:52.472102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:52.473731 (+ 1629us) server_negotiation.cc:407] Connection header received
0807 06:12:52.475544 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:52.475548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:52.475604 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:52.475683 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:52.476701 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.477319 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.478552 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.478789 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.478890 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:52.480362 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:52.480366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:52.480369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:52.480385 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:52.480411 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:52.480420 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:52.480481 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:52.480637 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 06:12:52.480708 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 06:12:52.482056 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:52.472444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35469 (local address 127.9.133.131:48093)
0807 06:12:52.473544 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:52.473609 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:52.475296 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:52.475846 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:52.475853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:52.475963 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:52.476570 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:52.476585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.477443 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.477445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:52.478431 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:52.478441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:52.478981 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:52.478984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:52.479293 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:52.480069 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:52.480071 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:52.480123 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:52.480205 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:52.480495 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:52.480502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:52.480507 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 06:12:52.480619 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:12:52.480946 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":189,"threads_started":1}
I20260807 06:12:52.483657 23564 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35469
I20260807 06:12:52.484030 23564 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:52.484795 23564 heartbeater.cc:511] Master 127.9.133.190:35469 requested a full tablet report, sending...
I20260807 06:12:52.486186 22863 ts_manager.cc:194] Registered new tserver with Master: 9b6aeec465d34b2198f0f00512d2e90a (127.9.133.131:42055)
I20260807 06:12:52.486914 22863 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.131:48093
I20260807 06:12:52.498873 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:12:52.502750 22921 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:52.501499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.9.133.190:35469)
0807 06:12:52.501651 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:52.501655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:52.501959 (+ 304us) server_negotiation.cc:407] Connection header received
0807 06:12:52.502072 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:52.502074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:52.502136 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:52.502227 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:52.502548 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60032: BlockingRecv error: recv got EOF from 127.0.0.1:60032 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 06:12:52.503173 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 22956
I20260807 06:12:52.525249 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 23154
I20260807 06:12:52.538075 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 23367
I20260807 06:12:52.554219 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 22816
2026-08-07T06:12:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1138 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T06:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:52Z Disabled control of system clock
I20260807 06:12:52.603739 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45923
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:45923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:12:52.738487 23576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:52.738847 23576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:52.738889 23576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:52.743448 23576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:52.743538 23576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:52.743556 23576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:52.743574 23576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:52.743590 23576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:52.749231 23576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:45923
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:52.750689 23576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:52.752568 23576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:52.762956 23584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:52.762964 23586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:52.763033 23583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:52.763504 23576 server_base.cc:1034] running on GCE node
I20260807 06:12:52.764163 23576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:52.765506 23576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:52.766732 23576 hybrid_clock.cc:648] HybridClock initialized: now 1786083172766710 us; error 46 us; skew 500 ppm
I20260807 06:12:52.769155 23576 webserver.cc:460] Webserver started at http://127.9.133.190:34191/ using document root <none> and password file <none>
I20260807 06:12:52.769876 23576 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:52.769985 23576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:52.770291 23576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:52.772965 23576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "9303e459d49c494b90279b34ba2b1791"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "7903c0f28c0ff4b254b569d5612e3a62"
server_key_iv: "5585c65ba5b1383f8b5ca06407a8a8cd"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.773669 23576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "9303e459d49c494b90279b34ba2b1791"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "7903c0f28c0ff4b254b569d5612e3a62"
server_key_iv: "5585c65ba5b1383f8b5ca06407a8a8cd"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.778201 23576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 06:12:52.781229 23595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:52.782872 23576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:12:52.783093 23576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "9303e459d49c494b90279b34ba2b1791"
format_stamp: "Formatted at 2026-08-07 06:12:52 on dist-test-slave-h3ft"
server_key: "7903c0f28c0ff4b254b569d5612e3a62"
server_key_iv: "5585c65ba5b1383f8b5ca06407a8a8cd"
server_key_version: "encryptionkey@0"
I20260807 06:12:52.783246 23576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:52.788596 23576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:52.789327 23576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:52.789535 23576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:52.798501 23576 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:45923
I20260807 06:12:52.798503 23676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:45923 every 8 connection(s)
I20260807 06:12:52.799774 23576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:52.802078 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 23576
I20260807 06:12:52.802240 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:52.802660 9750 external_mini_cluster.cc:1428] Setting key 5329ead8a625de987e9f43ff4b041048
I20260807 06:12:52.805002 23678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:52.810128 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:52.804743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.9.133.190:45923)
0807 06:12:52.805629 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:52.805648 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:52.805694 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:12:52.806485 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:52.806505 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:52.806946 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:52.807444 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:52.808522 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:52.808527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:52.808542 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:52.808553 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:52.808627 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:52.808646 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:52.808827 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:52.808908 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 06:12:52.809216 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":171,"threads_started":1}
I20260807 06:12:52.813364 23678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791: Bootstrap starting.
I20260807 06:12:52.816527 23678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:52.817497 23678 log.cc:824] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:52.820173 23678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791: No bootstrap required, opened a new log
I20260807 06:12:52.824391 23678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9303e459d49c494b90279b34ba2b1791" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45923 } }
I20260807 06:12:52.824664 23678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:52.824740 23678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9303e459d49c494b90279b34ba2b1791, State: Initialized, Role: FOLLOWER
I20260807 06:12:52.825325 23678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9303e459d49c494b90279b34ba2b1791" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45923 } }
I20260807 06:12:52.825490 23678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:52.825562 23678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:52.825675 23678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:52.826762 23678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9303e459d49c494b90279b34ba2b1791" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45923 } }
I20260807 06:12:52.827215 23678 leader_election.cc:302] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9303e459d49c494b90279b34ba2b1791; no voters:
I20260807 06:12:52.827596 23678 leader_election.cc:288] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:52.827762 23691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:52.828037 23691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [term 1 LEADER]: Becoming Leader. State: Replica: 9303e459d49c494b90279b34ba2b1791, State: Running, Role: LEADER
I20260807 06:12:52.828476 23691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9303e459d49c494b90279b34ba2b1791" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45923 } }
I20260807 06:12:52.829183 23678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:52.830581 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9303e459d49c494b90279b34ba2b1791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9303e459d49c494b90279b34ba2b1791" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45923 } } }
I20260807 06:12:52.830735 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:52.831058 23694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9303e459d49c494b90279b34ba2b1791. Latest consensus state: current_term: 1 leader_uuid: "9303e459d49c494b90279b34ba2b1791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9303e459d49c494b90279b34ba2b1791" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45923 } } }
I20260807 06:12:52.831159 23694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:52.831215 23702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:52.835215 23702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:52.841645 23702 catalog_manager.cc:1347] Generated new cluster ID: d9df563a057c4b77831854aa45a7b86a
I20260807 06:12:52.841742 23702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:52.858305 23702 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:52.858754 23702 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:52.865002 23702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9303e459d49c494b90279b34ba2b1791: Generated new TSK 0
I20260807 06:12:52.865943 23702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:12:52.871140 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45923
--builtin_ntp_servers=127.9.133.148:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 06:12:53.012293 23716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:53.012696 23716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:53.012779 23716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:53.016993 23716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:53.017099 23716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:53.017203 23716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:53.022331 23716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:53.023897 23716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:53.025779 23716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:53.038126 23724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:53.038126 23729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:53.038177 23726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:53.038427 23716 server_base.cc:1034] running on GCE node
I20260807 06:12:53.038995 23716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:53.039968 23716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:53.041174 23716 hybrid_clock.cc:648] HybridClock initialized: now 1786083173041146 us; error 36 us; skew 500 ppm
I20260807 06:12:53.043807 23716 webserver.cc:460] Webserver started at http://127.9.133.129:41751/ using document root <none> and password file <none>
I20260807 06:12:53.044489 23716 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:53.044556 23716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:53.044761 23716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:53.047365 23716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "8bc687cd9c5e44b890fa990ee20a3741"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "17a7c37ed6464c3af67ed0a685a31793"
server_key_iv: "f64f1b8a55a1f1561c5b841e10d77b86"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.048012 23716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "8bc687cd9c5e44b890fa990ee20a3741"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "17a7c37ed6464c3af67ed0a685a31793"
server_key_iv: "f64f1b8a55a1f1561c5b841e10d77b86"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.052526 23716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:53.055789 23738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.057523 23716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:12:53.057652 23716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "8bc687cd9c5e44b890fa990ee20a3741"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "17a7c37ed6464c3af67ed0a685a31793"
server_key_iv: "f64f1b8a55a1f1561c5b841e10d77b86"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.057775 23716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:53.063483 23716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:53.064258 23716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:53.064432 23716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:53.065066 23716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:53.066224 23716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:53.066313 23716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.066383 23716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:53.066413 23716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.077212 23716 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:41933
I20260807 06:12:53.077236 23930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:41933 every 8 connection(s)
I20260807 06:12:53.078389 23716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:53.082592 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 23716
I20260807 06:12:53.082764 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:53.083200 9750 external_mini_cluster.cc:1428] Setting key 3d8de954fc6c6610dc54fa8caf893db9
I20260807 06:12:53.085743 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45923
--builtin_ntp_servers=127.9.133.148:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 06:12:53.091638 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:53.080775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:44291 (local address 127.9.133.190:45923)
0807 06:12:53.080969 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:53.080974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:53.082350 (+ 1376us) server_negotiation.cc:407] Connection header received
0807 06:12:53.084082 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:53.084088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:53.084144 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:53.084232 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:53.090895 (+ 6663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:53.090910 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:53.090917 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:53.090943 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:53.090984 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:53.091050 (+ 66us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:53.091197 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:53.091350 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 06:12:53.091421 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:12:53.092934 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:53.081341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45923 (local address 127.9.133.129:44291)
0807 06:12:53.082161 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:53.082232 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:53.083841 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:53.084481 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:53.084491 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:53.084619 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:53.090442 (+ 5823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:53.090444 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:53.090518 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:53.090650 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:53.091167 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:53.091178 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:53.091185 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:12:53.091414 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 06:12:53.091838 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":146,"threads_started":1}
I20260807 06:12:53.094565 23931 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45923
I20260807 06:12:53.094882 23931 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:53.095491 23931 heartbeater.cc:511] Master 127.9.133.190:45923 requested a full tablet report, sending...
I20260807 06:12:53.097525 23622 ts_manager.cc:194] Registered new tserver with Master: 8bc687cd9c5e44b890fa990ee20a3741 (127.9.133.129:41933)
W20260807 06:12:53.220728 23934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:53.221093 23934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:53.221138 23934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:53.225435 23934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:53.225517 23934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:53.225631 23934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:53.231174 23934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:53.232762 23934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:53.234651 23934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:53.246438 23943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:53.246483 23946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:53.246711 23934 server_base.cc:1034] running on GCE node
W20260807 06:12:53.246483 23942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:53.247454 23934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:53.248340 23934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:53.249568 23934 hybrid_clock.cc:648] HybridClock initialized: now 1786083173249550 us; error 33 us; skew 500 ppm
I20260807 06:12:53.252184 23934 webserver.cc:460] Webserver started at http://127.9.133.130:45341/ using document root <none> and password file <none>
I20260807 06:12:53.253006 23934 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:53.253120 23934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:53.253418 23934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:53.256289 23934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "60e10bd218be4cb8b59073a6fe3a1798"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "13b6fbeee7ca36a294b5d03c3d1e5519"
server_key_iv: "f1352cc36e68b28143634992bbb1d87c"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.257042 23934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "60e10bd218be4cb8b59073a6fe3a1798"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "13b6fbeee7ca36a294b5d03c3d1e5519"
server_key_iv: "f1352cc36e68b28143634992bbb1d87c"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.262053 23934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:12:53.265591 23958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.267503 23934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:53.267685 23934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "60e10bd218be4cb8b59073a6fe3a1798"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "13b6fbeee7ca36a294b5d03c3d1e5519"
server_key_iv: "f1352cc36e68b28143634992bbb1d87c"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.267851 23934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:53.273456 23934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:53.274226 23934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:53.274449 23934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:53.275251 23934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:53.276418 23934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:53.276496 23934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.276571 23934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:53.276624 23934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.287859 23934 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:44895
I20260807 06:12:53.287949 24117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:44895 every 8 connection(s)
I20260807 06:12:53.289076 23934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:53.295861 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 23934
I20260807 06:12:53.296036 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:53.296484 9750 external_mini_cluster.cc:1428] Setting key 399cd1c4cde01c88be9ffa1617347f33
I20260807 06:12:53.297083 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:53.291572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:35055 (local address 127.9.133.190:45923)
0807 06:12:53.291791 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:53.291796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:53.293261 (+ 1465us) server_negotiation.cc:407] Connection header received
0807 06:12:53.294786 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:53.294789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:53.294846 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:53.294948 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:53.296487 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:53.296491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:53.296498 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:53.296516 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:53.296548 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:53.296563 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:53.296677 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:53.296834 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 06:12:53.296897 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 06:12:53.298322 24121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:53.292129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45923 (local address 127.9.133.130:35055)
0807 06:12:53.293055 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:53.293117 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:53.294550 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:53.295211 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:53.295220 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:53.295326 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:53.296189 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:53.296191 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:53.296243 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:53.296324 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:53.296659 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:53.296669 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:53.296682 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 06:12:53.296844 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 06:12:53.297158 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":174,"threads_started":1}
I20260807 06:12:53.299391 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45923
--builtin_ntp_servers=127.9.133.148:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 06:12:53.299742 24119 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45923
I20260807 06:12:53.300050 24119 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:53.300578 24119 heartbeater.cc:511] Master 127.9.133.190:45923 requested a full tablet report, sending...
I20260807 06:12:53.301832 23622 ts_manager.cc:194] Registered new tserver with Master: 60e10bd218be4cb8b59073a6fe3a1798 (127.9.133.130:44895)
W20260807 06:12:53.437055 24123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:53.437434 24123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:53.437536 24123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:53.441915 24123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:53.442039 24123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:53.442154 24123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:53.447535 24123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:53.449111 24123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:53.450966 24123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:53.463234 24132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:53.463231 24129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:53.463222 24134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:53.463562 24123 server_base.cc:1034] running on GCE node
I20260807 06:12:53.464139 24123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:53.464898 24123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:53.466085 24123 hybrid_clock.cc:648] HybridClock initialized: now 1786083173466061 us; error 37 us; skew 500 ppm
I20260807 06:12:53.468612 24123 webserver.cc:460] Webserver started at http://127.9.133.131:34823/ using document root <none> and password file <none>
I20260807 06:12:53.469362 24123 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:53.469437 24123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:53.469714 24123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:53.472570 24123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "308fffc1bee54c0f8784fa807923a3b5"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "e1abde8b01426e59c1feaff2c7857cea"
server_key_iv: "ee7a819bbb7edbf0056601a57f071a79"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.473361 24123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "308fffc1bee54c0f8784fa807923a3b5"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "e1abde8b01426e59c1feaff2c7857cea"
server_key_iv: "ee7a819bbb7edbf0056601a57f071a79"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.478122 24123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:53.481544 24140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.483223 24123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:53.483376 24123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "308fffc1bee54c0f8784fa807923a3b5"
format_stamp: "Formatted at 2026-08-07 06:12:53 on dist-test-slave-h3ft"
server_key: "e1abde8b01426e59c1feaff2c7857cea"
server_key_iv: "ee7a819bbb7edbf0056601a57f071a79"
server_key_version: "encryptionkey@0"
I20260807 06:12:53.483523 24123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:53.489243 24123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:53.489980 24123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:53.490200 24123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:53.490860 24123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:53.492080 24123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:53.492157 24123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.492231 24123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:53.492282 24123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:53.503605 24123 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:42721
I20260807 06:12:53.503630 24310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:42721 every 8 connection(s)
I20260807 06:12:53.504791 24123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:53.509233 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 24123
I20260807 06:12:53.509377 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:53.509768 9750 external_mini_cluster.cc:1428] Setting key cb81f4a12b684473ebd485d8edaf56c0
I20260807 06:12:53.512869 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:53.507200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:41951 (local address 127.9.133.190:45923)
0807 06:12:53.507379 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:53.507383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:53.508667 (+ 1284us) server_negotiation.cc:407] Connection header received
0807 06:12:53.510696 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:53.510700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:53.510757 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:53.510873 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:53.512312 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:53.512316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:53.512319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:12:53.512331 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:53.512358 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:12:53.512376 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:53.512475 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:53.512647 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 06:12:53.512706 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:12:53.513942 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:53.507692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45923 (local address 127.9.133.131:41951)
0807 06:12:53.508478 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:53.508525 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:53.510448 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:53.511136 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:53.511147 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:53.511291 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:53.512017 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:12:53.512018 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:53.512075 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:12:53.512157 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:53.512463 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:53.512472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:53.512479 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:12:53.512706 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 06:12:53.513017 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":151,"threads_started":1}
I20260807 06:12:53.515151 24311 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45923
I20260807 06:12:53.515421 24311 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:53.515969 24311 heartbeater.cc:511] Master 127.9.133.190:45923 requested a full tablet report, sending...
I20260807 06:12:53.517130 23622 ts_manager.cc:194] Registered new tserver with Master: 308fffc1bee54c0f8784fa807923a3b5 (127.9.133.131:42721)
I20260807 06:12:53.525281 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 06:12:53.528934 23689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:12:53.527884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.9.133.190:45923)
0807 06:12:53.528103 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:53.528107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:53.528221 (+ 114us) server_negotiation.cc:407] Connection header received
0807 06:12:53.528343 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:53.528346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:53.528385 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:53.528453 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:53.528709 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54874: BlockingRecv error: recv got EOF from 127.0.0.1:54874 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:12:53.528985 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 23716
I20260807 06:12:53.553439 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 23934
I20260807 06:12:53.570292 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 24123
I20260807 06:12:53.589421 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 23576
2026-08-07T06:12:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (1030 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:53 dist-test-slave-h3ft krb5kdc[24325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:53 dist-test-slave-h3ft krb5kdc[24325](info): set up 2 sockets
Aug 07 06:12:53 dist-test-slave-h3ft krb5kdc[24325](info): commencing operation
krb5kdc: starting...
W20260807 06:12:54.891480 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:54 dist-test-slave-h3ft krb5kdc[24325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:12:55.047694 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36293
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:55.189433 24353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:55.189783 24353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:55.189829 24353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:55.194128 24353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:12:55.194207 24353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:55.194227 24353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:55.194245 24353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:12:55.194262 24353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:12:55.200161 24353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:36293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:55.201593 24353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:55.202894 24353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:55.212953 24366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:55.212985 24360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:55.213001 24361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:55.213407 24353 server_base.cc:1034] running on GCE node
I20260807 06:12:55.214138 24353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:55.215500 24353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:55.216687 24353 hybrid_clock.cc:648] HybridClock initialized: now 1786083175216635 us; error 72 us; skew 500 ppm
Aug 07 06:12:55 dist-test-slave-h3ft krb5kdc[24325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083175, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:55.219915 24353 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:12:55.220950 24353 webserver.cc:460] Webserver started at http://127.9.133.190:34409/ using document root <none> and password file <none>
I20260807 06:12:55.221581 24353 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:55.221642 24353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:55.221843 24353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:55.224321 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "4eea04bb3c5a43fbab997272c774d696"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "4e289b99ead03bf8e697f984d8447b32"
server_key_iv: "d9189d0a935f957bac55edef46553cd3"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.224913 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "4eea04bb3c5a43fbab997272c774d696"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "4e289b99ead03bf8e697f984d8447b32"
server_key_iv: "d9189d0a935f957bac55edef46553cd3"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.229373 24353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:12:55.232340 24376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.234158 24353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:55.234375 24353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "4eea04bb3c5a43fbab997272c774d696"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "4e289b99ead03bf8e697f984d8447b32"
server_key_iv: "d9189d0a935f957bac55edef46553cd3"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.234546 24353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:55.256630 24353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:55.260878 24353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:55.261091 24353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:55.270150 24353 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:36293
I20260807 06:12:55.270511 24452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:36293 every 8 connection(s)
I20260807 06:12:55.271553 24353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:12:55.275563 24454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:55.278415 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 24353
I20260807 06:12:55.278580 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:12:55.278939 9750 external_mini_cluster.cc:1428] Setting key 6402b1b3c0fa11d2ccbdd3aef26e5118
I20260807 06:12:55.283802 24454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696: Bootstrap starting.
I20260807 06:12:55.287213 24454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:55.288365 24454 log.cc:824] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:12:55 dist-test-slave-h3ft krb5kdc[24325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:55.291172 24454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696: No bootstrap required, opened a new log
I20260807 06:12:55.296839 24454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eea04bb3c5a43fbab997272c774d696" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36293 } }
I20260807 06:12:55.297194 24454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:55.297248 24454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea04bb3c5a43fbab997272c774d696, State: Initialized, Role: FOLLOWER
I20260807 06:12:55.297816 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:55.280871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.9.133.190:36293)
0807 06:12:55.281420 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:55.281443 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:55.281497 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:12:55.282444 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:55.282477 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:55.282982 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:55.283464 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:55.284451 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.286057 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.286963 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.287778 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.287953 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:55.291525 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:55.291532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:55.291552 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:55.291590 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:55.293951 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:55.294483 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:55.294491 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:55.294498 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:55.294670 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:55.295038 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:55.295042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:55.295044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:55.295667 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:55.295919 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:55.296262 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 06:12:55.296627 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":156,"threads_started":1}
I20260807 06:12:55.297843 24454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eea04bb3c5a43fbab997272c774d696" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36293 } }
I20260807 06:12:55.298017 24454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:12:55.298139 24454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:12:55.298273 24454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:55.299510 24454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eea04bb3c5a43fbab997272c774d696" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36293 } }
I20260807 06:12:55.299923 24454 leader_election.cc:302] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eea04bb3c5a43fbab997272c774d696; no voters:
I20260807 06:12:55.300338 24454 leader_election.cc:288] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:12:55.300431 24460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:55.300630 24460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [term 1 LEADER]: Becoming Leader. State: Replica: 4eea04bb3c5a43fbab997272c774d696, State: Running, Role: LEADER
I20260807 06:12:55.301054 24460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eea04bb3c5a43fbab997272c774d696" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36293 } }
I20260807 06:12:55.301914 24454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:12:55.303433 24462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eea04bb3c5a43fbab997272c774d696. Latest consensus state: current_term: 1 leader_uuid: "4eea04bb3c5a43fbab997272c774d696" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eea04bb3c5a43fbab997272c774d696" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36293 } } }
I20260807 06:12:55.303580 24462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:55.303932 24461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eea04bb3c5a43fbab997272c774d696" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eea04bb3c5a43fbab997272c774d696" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36293 } } }
I20260807 06:12:55.304596 24477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:12:55.304689 24461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696 [sys.catalog]: This master's current role is: LEADER
I20260807 06:12:55.308094 24477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:12:55.314321 24477 catalog_manager.cc:1347] Generated new cluster ID: 9cdaee0a380c4518a6ff75ea4271fa3d
I20260807 06:12:55.314409 24477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:12:55.332238 24477 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:12:55.333853 24477 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:12:55.341990 24477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696: Generated new TSK 0
I20260807 06:12:55.343070 24477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:12:55.380107 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36293
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:55.528280 24494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:55.528649 24494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:55.528705 24494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:55.534788 24494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:55.534912 24494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:55.535097 24494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:12:55.541834 24494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:55.543380 24494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:55.544773 24494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:12:55.557274 24494 server_base.cc:1034] running on GCE node
W20260807 06:12:55.557178 24502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:55.557178 24501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:55.557178 24505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:55.557927 24494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:55.558764 24494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:55.559978 24494 hybrid_clock.cc:648] HybridClock initialized: now 1786083175559942 us; error 44 us; skew 500 ppm
Aug 07 06:12:55 dist-test-slave-h3ft krb5kdc[24325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083175, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:55.563787 24494 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:12:55.564888 24494 webserver.cc:460] Webserver started at http://127.9.133.129:43307/ using document root <none> and password file <none>
I20260807 06:12:55.565573 24494 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:55.565644 24494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:55.565861 24494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:55.568506 24494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "f652f8cc70de4b87aca70d5fc23d0569"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "2c36e5316810c6ebb95d9e1eb012d384"
server_key_iv: "ab77f9c757e4a4b704a77d7962b21c44"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.569130 24494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "f652f8cc70de4b87aca70d5fc23d0569"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "2c36e5316810c6ebb95d9e1eb012d384"
server_key_iv: "ab77f9c757e4a4b704a77d7962b21c44"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.574182 24494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:12:55.577448 24514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.579090 24494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:12:55.579255 24494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "f652f8cc70de4b87aca70d5fc23d0569"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "2c36e5316810c6ebb95d9e1eb012d384"
server_key_iv: "ab77f9c757e4a4b704a77d7962b21c44"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.579396 24494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:55.599705 24494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:55.604286 24494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:55.604559 24494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:55.605477 24494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:55.606602 24494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:55.606684 24494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.606768 24494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:55.606813 24494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.617496 24494 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:33479
I20260807 06:12:55.617523 24685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:33479 every 8 connection(s)
I20260807 06:12:55.618700 24494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:12:55.620483 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 24494
I20260807 06:12:55.620633 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:12:55.620987 9750 external_mini_cluster.cc:1428] Setting key 061ccf1b423aecc19377b4349a38f9ae
Aug 07 06:12:55 dist-test-slave-h3ft krb5kdc[24325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083175, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:12:55.636161 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:55.621304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:38591 (local address 127.9.133.190:36293)
0807 06:12:55.621513 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:55.621518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:55.622698 (+ 1180us) server_negotiation.cc:407] Connection header received
0807 06:12:55.624374 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:55.624378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:55.624440 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:55.624535 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:55.625940 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.626561 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.627927 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.628207 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.628333 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:55.631412 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:55.631416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:55.631421 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:55.631456 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:55.633347 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:55.633957 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:55.633961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:55.633965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:55.634076 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:55.634585 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:55.634588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:55.634589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:55.634828 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:55.634915 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:55.635834 (+ 919us) server_negotiation.cc:299] Negotiation successful
0807 06:12:55.635992 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 06:12:55.637315 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:55.621909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36293 (local address 127.9.133.129:38591)
0807 06:12:55.622524 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:55.622589 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:55.624130 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:55.624737 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:55.624745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:55.625192 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:55.625774 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:55.625791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.626715 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.626719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:55.627786 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:55.627798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.628390 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.628394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:55.628594 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:55.629341 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:55.629361 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:55.631199 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:55.633484 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:55.633490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:55.633506 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:55.633821 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:55.634227 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:55.634231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:55.634233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:55.634455 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:55.634981 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:55.634989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:55.635522 (+ 533us) client_negotiation.cc:769] Sending connection context
0807 06:12:55.635744 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 06:12:55.636132 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":182,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:12:55.639248 24686 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36293
I20260807 06:12:55.639645 24686 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:55.640534 24686 heartbeater.cc:511] Master 127.9.133.190:36293 requested a full tablet report, sending...
I20260807 06:12:55.643222 24403 ts_manager.cc:194] Registered new tserver with Master: f652f8cc70de4b87aca70d5fc23d0569 (127.9.133.129:33479)
I20260807 06:12:55.645504 24403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:38591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:12:55.664139 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36293
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:55.806557 24698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:55.806891 24698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:55.806948 24698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:55.811177 24698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:55.811239 24698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:55.811304 24698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:12:55.816875 24698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:55.818176 24698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:55.819489 24698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:55.831300 24708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:55.831272 24711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:55.831336 24698 server_base.cc:1034] running on GCE node
W20260807 06:12:55.831264 24716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:55.832054 24698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:55.832907 24698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:55.834117 24698 hybrid_clock.cc:648] HybridClock initialized: now 1786083175834099 us; error 33 us; skew 500 ppm
Aug 07 06:12:55 dist-test-slave-h3ft krb5kdc[24325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083175, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:55.837615 24698 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:12:55.838739 24698 webserver.cc:460] Webserver started at http://127.9.133.130:42535/ using document root <none> and password file <none>
I20260807 06:12:55.839479 24698 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:55.839547 24698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:55.839749 24698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:55.842406 24698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "08e323cedd944a6e810967a1d75596cf"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "86f492ed5a33685bbdeae905be7dc897"
server_key_iv: "5fb86f6a63f68d08f53c0a202cb5ee7c"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.843099 24698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "08e323cedd944a6e810967a1d75596cf"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "86f492ed5a33685bbdeae905be7dc897"
server_key_iv: "5fb86f6a63f68d08f53c0a202cb5ee7c"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.847543 24698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:12:55.850515 24723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.852226 24698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:12:55.852339 24698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "08e323cedd944a6e810967a1d75596cf"
format_stamp: "Formatted at 2026-08-07 06:12:55 on dist-test-slave-h3ft"
server_key: "86f492ed5a33685bbdeae905be7dc897"
server_key_iv: "5fb86f6a63f68d08f53c0a202cb5ee7c"
server_key_version: "encryptionkey@0"
I20260807 06:12:55.852442 24698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:55.868207 24698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:55.872148 24698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:55.872332 24698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:55.873018 24698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:55.874082 24698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:55.874135 24698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.874181 24698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:55.874203 24698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:55.885512 24698 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:44431
I20260807 06:12:55.885550 24907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:44431 every 8 connection(s)
I20260807 06:12:55.886701 24698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:12:55.895048 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 24698
I20260807 06:12:55.895200 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:12:55.895601 9750 external_mini_cluster.cc:1428] Setting key acdeb8c77019427197c0c32f9457e2bd
Aug 07 06:12:55 dist-test-slave-h3ft krb5kdc[24325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083175, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:55.904577 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:55.889213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:52833 (local address 127.9.133.190:36293)
0807 06:12:55.889390 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:55.889394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:55.890478 (+ 1084us) server_negotiation.cc:407] Connection header received
0807 06:12:55.892072 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:55.892076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:55.892142 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:55.892249 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:55.893712 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.894379 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.896283 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.896608 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.896730 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:55.899982 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:55.899989 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:55.899995 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:55.900037 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:55.901723 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:55.902434 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:55.902437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:55.902439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:55.902534 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:55.903022 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:55.903026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:55.903029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:55.903264 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:55.903347 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:55.904154 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 06:12:55.904304 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 06:12:55.906003 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:55.889720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36293 (local address 127.9.133.130:52833)
0807 06:12:55.890283 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:55.890334 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:55.891817 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:55.892437 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:55.892444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:55.892903 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:55.893528 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:55.893544 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.894532 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.894535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:55.896134 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:55.896146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:55.896769 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:55.896772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:55.896932 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:55.897706 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:55.897726 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:55.899772 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:55.901853 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:55.901864 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:55.901886 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:55.902309 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:55.902629 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:55.902635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:55.902641 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:55.902856 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:55.903387 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:55.903397 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:55.903800 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 06:12:55.904069 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 06:12:55.904557 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":157,"threads_started":1}
I20260807 06:12:55.907636 24908 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36293
I20260807 06:12:55.907963 24908 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:55.908624 24908 heartbeater.cc:511] Master 127.9.133.190:36293 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:12:55.909983 24403 ts_manager.cc:194] Registered new tserver with Master: 08e323cedd944a6e810967a1d75596cf (127.9.133.130:44431)
I20260807 06:12:55.910692 24403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:52833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:12:55.935381 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36293
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:12:56.090529 24916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:12:56.091084 24916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:12:56.091197 24916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:12:56.101490 24916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:12:56.101630 24916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:12:56.101794 24916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:12:56.115146 24916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:12:56.117338 24916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:12:56.119213 24916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:12:56.132201 24925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:56.132197 24926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:12:56.132238 24929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:12:56.132654 24916 server_base.cc:1034] running on GCE node
I20260807 06:12:56.133236 24916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:12:56.134100 24916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:12:56.135342 24916 hybrid_clock.cc:648] HybridClock initialized: now 1786083176135310 us; error 60 us; skew 500 ppm
Aug 07 06:12:56 dist-test-slave-h3ft krb5kdc[24325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083176, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:12:56.139361 24916 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:12:56.140877 24916 webserver.cc:460] Webserver started at http://127.9.133.131:42503/ using document root <none> and password file <none>
I20260807 06:12:56.141832 24916 fs_manager.cc:359] Metadata directory not provided
I20260807 06:12:56.141952 24916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:12:56.142262 24916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:12:56.146144 24916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "665bdfd4b07c43a09ad40947def9c29f"
format_stamp: "Formatted at 2026-08-07 06:12:56 on dist-test-slave-h3ft"
server_key: "229ce1f84b491f0a5f0b7cba42d1b13b"
server_key_iv: "6f33d7e229e3d10490266a64529dcd0a"
server_key_version: "encryptionkey@0"
I20260807 06:12:56.147089 24916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "665bdfd4b07c43a09ad40947def9c29f"
format_stamp: "Formatted at 2026-08-07 06:12:56 on dist-test-slave-h3ft"
server_key: "229ce1f84b491f0a5f0b7cba42d1b13b"
server_key_iv: "6f33d7e229e3d10490266a64529dcd0a"
server_key_version: "encryptionkey@0"
I20260807 06:12:56.153644 24916 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 06:12:56.157735 24941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:56.159647 24916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:12:56.159802 24916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "665bdfd4b07c43a09ad40947def9c29f"
format_stamp: "Formatted at 2026-08-07 06:12:56 on dist-test-slave-h3ft"
server_key: "229ce1f84b491f0a5f0b7cba42d1b13b"
server_key_iv: "6f33d7e229e3d10490266a64529dcd0a"
server_key_version: "encryptionkey@0"
I20260807 06:12:56.159960 24916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:12:56.181057 24916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:12:56.185354 24916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:12:56.185612 24916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:12:56.186384 24916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:12:56.187650 24916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:12:56.187705 24916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:56.187753 24916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:12:56.187808 24916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:12:56.200130 24916 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:44597
I20260807 06:12:56.200213 25112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:44597 every 8 connection(s)
I20260807 06:12:56.201295 24916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:12:56.206759 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 24916
I20260807 06:12:56.206911 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:12:56.207345 9750 external_mini_cluster.cc:1428] Setting key 08b6cbd2616335207521569068fb9b11
Aug 07 06:12:56 dist-test-slave-h3ft krb5kdc[24325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083176, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:12:56.219717 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.203990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:57543 (local address 127.9.133.190:36293)
0807 06:12:56.204181 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.204185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.205559 (+ 1374us) server_negotiation.cc:407] Connection header received
0807 06:12:56.207607 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.207612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.207688 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.207752 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:56.209757 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.210489 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.211879 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.212121 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.212242 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.215416 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:56.215420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:56.215422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:56.215450 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:56.217229 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:56.217852 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:56.217856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:56.217858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:56.217956 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:56.218358 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:56.218368 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:56.218370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:56.218581 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:56.218699 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.219367 (+ 668us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.219512 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 06:12:56.221040 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.204703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36293 (local address 127.9.133.131:57543)
0807 06:12:56.205360 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.205415 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:56.207285 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:56.208244 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:56.208256 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:56.208906 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:12:56.209608 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.209623 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.210616 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.210622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.211746 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.211756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.212288 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.212293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.212490 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.213384 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:12:56.213404 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:12:56.215221 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:12:56.217406 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:56.217412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:56.217430 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:56.217730 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:56.218061 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:12:56.218064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:12:56.218066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:12:56.218228 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:12:56.218695 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:12:56.218702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:12:56.219077 (+ 375us) client_negotiation.cc:769] Sending connection context
0807 06:12:56.219295 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 06:12:56.219681 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":185,"threads_started":1}
I20260807 06:12:56.222573 25113 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36293
I20260807 06:12:56.222859 25113 heartbeater.cc:464] Registering TS with master...
I20260807 06:12:56.223538 25113 heartbeater.cc:511] Master 127.9.133.190:36293 requested a full tablet report, sending...
I20260807 06:12:56.224809 24403 ts_manager.cc:194] Registered new tserver with Master: 665bdfd4b07c43a09ad40947def9c29f (127.9.133.131:44597)
I20260807 06:12:56.225826 24403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:57543
I20260807 06:12:56.234987 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:12:56.248171 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.237520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.9.133.190:36293)
0807 06:12:56.237703 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.237711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.237916 (+ 205us) server_negotiation.cc:407] Connection header received
0807 06:12:56.238076 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.238083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.238151 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.238254 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:12:56.239262 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.240327 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.241208 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.241688 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.241814 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.243609 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:12:56.243616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:12:56.243621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:12:56.243657 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:12:56.245792 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:56.246320 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:56.246325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:56.246329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:56.246459 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:12:56.246829 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:12:56.246834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:12:56.246839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:12:56.247283 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:12:56.247457 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.247722 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.247888 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:12:56.252765 24403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:12:56.259056 24403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:12:56.279821 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.271436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33479 (local address 127.0.0.1:48916)
0807 06:12:56.272202 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.272215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:56.272299 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:56.274040 (+ 1741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:56.274044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:56.274065 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:56.274301 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.274311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.276314 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.276320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.277966 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.277977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.279213 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.279216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.279330 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.279400 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 06:12:56.279498 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:12:56.279623 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":141,"thread_start_us":140,"threads_started":1}
I20260807 06:12:56.280494 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.271311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:44597 (local address 127.0.0.1:52582)
0807 06:12:56.271738 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.271796 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:56.271950 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:56.273203 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:56.273212 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:56.273240 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:56.273446 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.273455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.275889 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.275893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.277485 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.277495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.279920 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.279926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.280079 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.280123 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 06:12:56.280180 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:12:56.280276 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":48,"thread_start_us":118,"threads_started":1}
I20260807 06:12:56.281019 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.272353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.9.133.129:33479)
0807 06:12:56.273496 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.273507 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.273537 (+ 30us) server_negotiation.cc:407] Connection header received
0807 06:12:56.273632 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.273640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.273854 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.274522 (+ 668us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:56.274726 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.276148 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.278115 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.279060 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.280415 (+ 1355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.280565 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.280726 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.280846 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":984,"thread_start_us":143,"threads_started":1}
I20260807 06:12:56.281423 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.274034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.9.133.130:44431)
0807 06:12:56.274772 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.274784 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.275276 (+ 492us) server_negotiation.cc:407] Connection header received
0807 06:12:56.275499 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.275505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.275681 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.275817 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:56.276304 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.277777 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.279205 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.280293 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.280386 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.280554 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.281059 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.281260 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":381,"threads_started":1}
I20260807 06:12:56.281502 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.272152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.9.133.131:44597)
0807 06:12:56.272611 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.272627 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.272655 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:12:56.272752 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.272757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.272954 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.274239 (+ 1285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:56.274452 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.275706 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.278633 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.279756 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.280859 (+ 1103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.281046 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.281107 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.281239 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":238,"threads_started":1}
I20260807 06:12:56.281821 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.273542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:44431 (local address 127.0.0.1:51404)
0807 06:12:56.275168 (+ 1626us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.275182 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:56.275358 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:56.276039 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:56.276042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:56.276061 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:56.276170 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.276176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.277918 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.277922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.279074 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.279085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.280792 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.280795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.280892 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.280916 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:12:56.281622 (+ 706us) client_negotiation.cc:240] Negotiation successful
0807 06:12:56.281702 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1562,"thread_start_us":118,"threads_started":1}
I20260807 06:12:56.284901 25015 tablet_service.cc:1467] Processing CreateTablet for tablet f9db16a0c1694ad9853b763af5d2e6b4 (DEFAULT_TABLE table=test-table [id=6377c3db18464fb2bacef40009692c7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:56.284896 24588 tablet_service.cc:1467] Processing CreateTablet for tablet f9db16a0c1694ad9853b763af5d2e6b4 (DEFAULT_TABLE table=test-table [id=6377c3db18464fb2bacef40009692c7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:56.286306 25015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9db16a0c1694ad9853b763af5d2e6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:56.286298 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9db16a0c1694ad9853b763af5d2e6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:56.286531 24807 tablet_service.cc:1467] Processing CreateTablet for tablet f9db16a0c1694ad9853b763af5d2e6b4 (DEFAULT_TABLE table=test-table [id=6377c3db18464fb2bacef40009692c7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:12:56.288353 24807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9db16a0c1694ad9853b763af5d2e6b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:12:56.296357 25138 tablet_bootstrap.cc:492] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: Bootstrap starting.
I20260807 06:12:56.296345 25141 tablet_bootstrap.cc:492] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Bootstrap starting.
I20260807 06:12:56.298482 25142 tablet_bootstrap.cc:492] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Bootstrap starting.
I20260807 06:12:56.299220 25138 tablet_bootstrap.cc:654] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:56.299968 25141 tablet_bootstrap.cc:654] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:56.300400 25138 log.cc:824] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:56.300915 25141 log.cc:824] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:56.301824 25142 tablet_bootstrap.cc:654] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Neither blocks nor log segments found. Creating new log.
I20260807 06:12:56.302974 25141 tablet_bootstrap.cc:492] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: No bootstrap required, opened a new log
I20260807 06:12:56.302981 25142 log.cc:824] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Log is configured to *not* fsync() on all Append() calls
I20260807 06:12:56.302974 25138 tablet_bootstrap.cc:492] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: No bootstrap required, opened a new log
I20260807 06:12:56.303242 25138 ts_tablet_manager.cc:1397] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260807 06:12:56.303243 25141 ts_tablet_manager.cc:1397] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 06:12:56.305706 25142 tablet_bootstrap.cc:492] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: No bootstrap required, opened a new log
I20260807 06:12:56.305927 25142 ts_tablet_manager.cc:1397] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 06:12:56.307953 25141 raft_consensus.cc:348] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.308284 25141 raft_consensus.cc:374] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:56.308337 25141 raft_consensus.cc:729] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665bdfd4b07c43a09ad40947def9c29f, State: Initialized, Role: FOLLOWER
I20260807 06:12:56.308184 25138 raft_consensus.cc:348] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.308435 25138 raft_consensus.cc:374] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:56.308506 25138 raft_consensus.cc:729] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f652f8cc70de4b87aca70d5fc23d0569, State: Initialized, Role: FOLLOWER
I20260807 06:12:56.308950 25141 consensus_queue.cc:260] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.308950 25138 consensus_queue.cc:260] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.310154 25141 ts_tablet_manager.cc:1428] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260807 06:12:56.310154 25138 ts_tablet_manager.cc:1428] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260807 06:12:56.310020 25142 raft_consensus.cc:348] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.310268 25142 raft_consensus.cc:374] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:12:56.310333 25142 raft_consensus.cc:729] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e323cedd944a6e810967a1d75596cf, State: Initialized, Role: FOLLOWER
I20260807 06:12:56.310602 25113 heartbeater.cc:503] Master 127.9.133.190:36293 was elected leader, sending a full tablet report...
I20260807 06:12:56.310846 25142 consensus_queue.cc:260] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.311596 24908 heartbeater.cc:503] Master 127.9.133.190:36293 was elected leader, sending a full tablet report...
I20260807 06:12:56.311892 25142 ts_tablet_manager.cc:1428] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:12:56.312640 24686 heartbeater.cc:503] Master 127.9.133.190:36293 was elected leader, sending a full tablet report...
W20260807 06:12:56.370533 24688 tablet.cc:2369] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:56.388278 24909 tablet.cc:2369] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:12:56.452745 25114 tablet.cc:2369] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:12:56.466867 25147 raft_consensus.cc:482] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:12:56.467262 25147 raft_consensus.cc:504] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.468886 25147 leader_election.cc:288] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 665bdfd4b07c43a09ad40947def9c29f (127.9.133.131:44597), 08e323cedd944a6e810967a1d75596cf (127.9.133.130:44431)
I20260807 06:12:56.475224 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.469351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:52685 (local address 127.9.133.131:44597)
0807 06:12:56.469553 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.469558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.469847 (+ 289us) server_negotiation.cc:407] Connection header received
0807 06:12:56.470071 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.470075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.470142 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.470240 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:56.470773 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.471647 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.473748 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.474407 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.474561 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.474709 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.474932 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.475046 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:12:56.475224 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.469454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:49645 (local address 127.9.133.130:44431)
0807 06:12:56.469683 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.469687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.469923 (+ 236us) server_negotiation.cc:407] Connection header received
0807 06:12:56.470074 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.470078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.470146 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.470241 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:12:56.470773 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.471647 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.473748 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.474407 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.474597 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.474707 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.474920 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.475062 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 06:12:56.475327 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.469228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:44597 (local address 127.9.133.129:52685)
0807 06:12:56.469726 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.469745 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:56.469910 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:56.470277 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:56.470280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:56.470333 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:56.470642 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.470650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.471778 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.471782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.473587 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.473599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.474707 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.474711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.474799 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.474828 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:12:56.474900 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:12:56.475076 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":293,"spinlock_wait_cycles":11904,"thread_start_us":148,"threads_started":1}
I20260807 06:12:56.475327 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.469296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:44431 (local address 127.9.133.129:49645)
0807 06:12:56.469827 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.469839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:12:56.469956 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:12:56.470277 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:12:56.470280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:12:56.470332 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:12:56.470641 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.470650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.471778 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.471782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.473586 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:12:56.473599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.474617 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.474621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:12:56.474744 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.474780 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 06:12:56.474882 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 06:12:56.475076 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":279,"spinlock_wait_cycles":55168,"thread_start_us":128,"threads_started":1}
I20260807 06:12:56.476220 25042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9db16a0c1694ad9853b763af5d2e6b4" candidate_uuid: "f652f8cc70de4b87aca70d5fc23d0569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "665bdfd4b07c43a09ad40947def9c29f" is_pre_election: true
I20260807 06:12:56.476219 24841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9db16a0c1694ad9853b763af5d2e6b4" candidate_uuid: "f652f8cc70de4b87aca70d5fc23d0569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e323cedd944a6e810967a1d75596cf" is_pre_election: true
I20260807 06:12:56.476635 25042 raft_consensus.cc:2454] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f652f8cc70de4b87aca70d5fc23d0569 in term 0.
I20260807 06:12:56.476637 24841 raft_consensus.cc:2454] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f652f8cc70de4b87aca70d5fc23d0569 in term 0.
I20260807 06:12:56.477288 24519 leader_election.cc:302] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08e323cedd944a6e810967a1d75596cf, 665bdfd4b07c43a09ad40947def9c29f, f652f8cc70de4b87aca70d5fc23d0569; no voters:
I20260807 06:12:56.477684 25147 raft_consensus.cc:2781] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:12:56.477792 25147 raft_consensus.cc:482] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:12:56.477838 25147 raft_consensus.cc:3037] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:56.479280 25147 raft_consensus.cc:504] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.479707 25147 leader_election.cc:288] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [CANDIDATE]: Term 1 election: Requested vote from peers 665bdfd4b07c43a09ad40947def9c29f (127.9.133.131:44597), 08e323cedd944a6e810967a1d75596cf (127.9.133.130:44431)
I20260807 06:12:56.480162 24841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9db16a0c1694ad9853b763af5d2e6b4" candidate_uuid: "f652f8cc70de4b87aca70d5fc23d0569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08e323cedd944a6e810967a1d75596cf"
I20260807 06:12:56.480198 25042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9db16a0c1694ad9853b763af5d2e6b4" candidate_uuid: "f652f8cc70de4b87aca70d5fc23d0569" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "665bdfd4b07c43a09ad40947def9c29f"
I20260807 06:12:56.480329 24841 raft_consensus.cc:3037] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:56.480350 25042 raft_consensus.cc:3037] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:12:56.481606 25042 raft_consensus.cc:2454] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f652f8cc70de4b87aca70d5fc23d0569 in term 1.
I20260807 06:12:56.481693 24841 raft_consensus.cc:2454] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f652f8cc70de4b87aca70d5fc23d0569 in term 1.
I20260807 06:12:56.482018 24520 leader_election.cc:302] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08e323cedd944a6e810967a1d75596cf, 665bdfd4b07c43a09ad40947def9c29f, f652f8cc70de4b87aca70d5fc23d0569; no voters:
I20260807 06:12:56.482205 25147 raft_consensus.cc:2781] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:12:56.482483 25147 raft_consensus.cc:686] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 1 LEADER]: Becoming Leader. State: Replica: f652f8cc70de4b87aca70d5fc23d0569, State: Running, Role: LEADER
I20260807 06:12:56.483018 25147 consensus_queue.cc:237] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } }
I20260807 06:12:56.487277 24403 catalog_manager.cc:5495] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 reported cstate change: term changed from 0 to 1, leader changed from <none> to f652f8cc70de4b87aca70d5fc23d0569 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "f652f8cc70de4b87aca70d5fc23d0569" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f652f8cc70de4b87aca70d5fc23d0569" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 } health_report { overall_health: UNKNOWN } } }
I20260807 06:12:56.516745 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.512673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.9.133.129:33479)
0807 06:12:56.512855 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:12:56.512863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:12:56.512884 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:12:56.513014 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:12:56.513017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:12:56.513088 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:12:56.513227 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:12:56.513516 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.514073 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.515023 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:12:56.515265 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:12:56.515383 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:12:56.515746 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:12:56.516293 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:12:56.516455 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 06:12:56.516491 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 06:12:56.516558 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 06:12:56.524513 24841 raft_consensus.cc:1261] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 1 FOLLOWER]: Refusing update from remote peer f652f8cc70de4b87aca70d5fc23d0569: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:56.524513 25042 raft_consensus.cc:1261] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 1 FOLLOWER]: Refusing update from remote peer f652f8cc70de4b87aca70d5fc23d0569: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:12:56.525566 25147 consensus_queue.cc:1035] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e323cedd944a6e810967a1d75596cf" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 44431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:56.525826 25152 consensus_queue.cc:1035] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [LEADER]: Connected to new peer: Peer: permanent_uuid: "665bdfd4b07c43a09ad40947def9c29f" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 44597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:12:56.536372 25156 mvcc.cc:205] Tried to move back new op lower bound from 7315796691034021888 to 7315796690878275584. Current Snapshot: MvccSnapshot[applied={T|T < 7315796691034021888}]
I20260807 06:12:56.546859 24403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:56.547147 24403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:12:56.550264 24403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4eea04bb3c5a43fbab997272c774d696: Sending DeleteTablet for 3 replicas of tablet f9db16a0c1694ad9853b763af5d2e6b4
I20260807 06:12:56.551260 25015 tablet_service.cc:1514] Processing DeleteTablet for tablet f9db16a0c1694ad9853b763af5d2e6b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:56 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:52582
I20260807 06:12:56.551424 24807 tablet_service.cc:1514] Processing DeleteTablet for tablet f9db16a0c1694ad9853b763af5d2e6b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:56 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:51404
I20260807 06:12:56.551553 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet f9db16a0c1694ad9853b763af5d2e6b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:12:56 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:48916
I20260807 06:12:56.551930 25169 tablet_replica.cc:330] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: stopping tablet replica
I20260807 06:12:56.552196 25170 tablet_replica.cc:330] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569: stopping tablet replica
I20260807 06:12:56.552369 25168 tablet_replica.cc:330] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: stopping tablet replica
I20260807 06:12:56.552685 25170 raft_consensus.cc:2229] T f9db16a0c1694ad9853b763af5d2e6b4 P f652f8cc70de4b87aca70d5fc23d0569 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:12:56.552719 25168 raft_consensus.cc:2229] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:56.552948 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 24494
I20260807 06:12:56.553128 25168 raft_consensus.cc:2258] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:56.555511 25168 ts_tablet_manager.cc:1905] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:56.559777 25169 raft_consensus.cc:2229] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:12:56.559903 25168 ts_tablet_manager.cc:1918] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:56.560014 25168 log.cc:1196] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/f9db16a0c1694ad9853b763af5d2e6b4
I20260807 06:12:56.560263 25169 raft_consensus.cc:2258] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:12:56.560388 25168 ts_tablet_manager.cc:1939] T f9db16a0c1694ad9853b763af5d2e6b4 P 665bdfd4b07c43a09ad40947def9c29f: Deleting consensus metadata
I20260807 06:12:56.562047 24386 catalog_manager.cc:4841] TS 665bdfd4b07c43a09ad40947def9c29f (127.9.133.131:44597): tablet f9db16a0c1694ad9853b763af5d2e6b4 (table test-table [id=6377c3db18464fb2bacef40009692c7e]) successfully deleted
I20260807 06:12:56.562242 25169 ts_tablet_manager.cc:1905] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:12:56.564713 25169 ts_tablet_manager.cc:1918] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:12:56.564846 25169 log.cc:1196] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/f9db16a0c1694ad9853b763af5d2e6b4
I20260807 06:12:56.565173 25169 ts_tablet_manager.cc:1939] T f9db16a0c1694ad9853b763af5d2e6b4 P 08e323cedd944a6e810967a1d75596cf: Deleting consensus metadata
I20260807 06:12:56.566417 24385 catalog_manager.cc:4841] TS 08e323cedd944a6e810967a1d75596cf (127.9.133.130:44431): tablet f9db16a0c1694ad9853b763af5d2e6b4 (table test-table [id=6377c3db18464fb2bacef40009692c7e]) successfully deleted
W20260807 06:12:56.567620 24385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:33479 (error 108)
I20260807 06:12:56.568663 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 24698
I20260807 06:12:56.568926 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:12:56.568520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33479 (local address 127.0.0.1:48934)
0807 06:12:56.568686 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:12:56.568800 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 06:12:56.575898 24385 catalog_manager.cc:4584] TS f652f8cc70de4b87aca70d5fc23d0569 (127.9.133.129:33479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9db16a0c1694ad9853b763af5d2e6b4: Network error: Client connection negotiation failed: client connection to 127.9.133.129:33479: connect: Connection refused (error 111)
I20260807 06:12:56.582973 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 24916
I20260807 06:12:56.597509 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 24353
2026-08-07T06:12:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3009 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:12:56 dist-test-slave-h3ft krb5kdc[25174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:12:56 dist-test-slave-h3ft krb5kdc[25174](info): set up 2 sockets
Aug 07 06:12:56 dist-test-slave-h3ft krb5kdc[25174](info): commencing operation
krb5kdc: starting...
W20260807 06:12:57.884770 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:12:57 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:12:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/rangeradmin_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/rangeradmin_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/rangerlookup_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/rangerlookup_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
I20260807 06:12:58.096526 9750 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrsqWAA/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/postgres -l logfile start
2026-08-07 06:13:00.970 UTC [25224] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 06:13:00.970 UTC [25224] LOG: listening on IPv4 address "127.9.133.148", port 35129
2026-08-07 06:13:00.973 UTC [25224] LOG: listening on Unix socket "/tmp/.s.PGSQL.35129"
2026-08-07 06:13:01.001 UTC [25229] LOG: database system was shut down at 2026-08-07 06:12:59 UTC
2026-08-07 06:13:01.006 UTC [25224] LOG: database system is ready to accept connections
I20260807 06:13:02.170694 9750 mini_postgres.cc:96] Postgres bound to 35129
2026-08-07 06:13:02.182 UTC [25240] FATAL: database "slave" does not exist
127.9.133.148:35129 - accepting connections
I20260807 06:13:02.183641 9750 mini_ranger.cc:161] Starting Ranger...
I20260807 06:13:02.205185 9750 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 06:13:02.782048 9750 mini_ranger.cc:87] Created ranger Postgres database
I20260807 06:13:02.782198 9750 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-admin
2026-08-07 06:13:03,179 [I] DB FLAVOR :POSTGRES
2026-08-07 06:13:03,180 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:13:03,180 [I] Checking connection..
2026-08-07 06:13:03,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 06:13:03,454 [I] Checking connection passed.
2026-08-07 06:13:03,454 [I] --------- Verifying version history table ---------
2026-08-07 06:13:03,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:13:03,747 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:13:03,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:13:04,054 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:13:04,054 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 06:13:04,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 06:13:04.343 UTC [25344] WARNING: there is no transaction in progress
2026-08-07 06:13:04,352 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 06:13:04,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:13:04,627 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 06:13:04,628 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:13:04,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 06:13:04,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 06:13:05,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-07 06:13:05,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:13:05,838 [I] Table x_portal_user does not exist in database ranger
2026-08-07 06:13:05,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:13:06,140 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 06:13:06,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:13:06,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:13:06,759 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 06:13:06,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 06:13:07.905 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:07.925 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:07.947 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:07.965 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:07.986 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.006 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.071 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.080 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.096 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.109 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.122 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.139 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.150 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.158 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.166 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08.174 UTC [25511] WARNING: there is no transaction in progress
2026-08-07 06:13:08,917 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 06:13:08,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 06:13:09,194 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 06:13:09,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:13:09,484 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 06:13:09,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:13:09,762 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 06:13:09,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:13:10,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:13:10,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-07 06:13:10,657 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 06:13:10,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 06:13:10,950 [I] DB_PATCHES have already been applied
I20260807 06:13:10.958010 9750 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrsqWAA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 06:13:10.958104 9750 mini_ranger.cc:193] Using host: 127.9.133.148
I20260807 06:13:10.963882 9750 mini_ranger.cc:227] Ranger admin URL: http://127.9.133.148:44309
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:13:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 06:13:12.350615 9750 mini_ranger.cc:160] Time spent starting Ranger: real 10.167s user 0.002s sys 0.015s
Aug 07, 2026 6:13:35 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:13:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 6:13:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:13:35 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:13:36 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 06:13:36.966804 9750 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/rangerkms_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/rangerkms_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.133.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.133.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 06:13:37 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083217, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 06:13:37.078642 9750 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 06:13:37.093061 9750 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 06:13:38.092195 9750 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 06:13:38.092335 9750 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms
I20260807 06:13:38.092473 9750 mini_ranger_kms.cc:227] Using postgres at 127.9.133.148:35129
2026-08-07 06:13:38,391 [I] DB FLAVOR :POSTGRES
2026-08-07 06:13:38,391 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:13:38,391 [I] Checking connection
2026-08-07 06:13:38,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 06:13:38,674 [I] connection success
2026-08-07 06:13:38,674 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 06:13:38,675 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 06:13:38,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 06:13:38,945 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 06:13:38,945 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:13:38,945 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 06:13:38,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:35129/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 06:13:39,262 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 06:13:39.850920 9750 mini_ranger_kms.cc:310] Created kms service
I20260807 06:13:40.059603 9750 mini_ranger_kms.cc:326] Created kudu user
I20260807 06:13:40.121161 9750 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 06:13:40 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083217, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:40.581563 9750 mini_ranger_kms.cc:384] Added ranger policy
I20260807 06:13:40.581812 9750 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 06:13:40.581842 9750 mini_ranger_kms.cc:240] Using host: 127.9.133.148
I20260807 06:13:40.588079 9750 mini_ranger_kms.cc:276] Ranger KMS PID: 25784
I20260807 06:13:40.588286 9750 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.133.148:40941
Aug 07, 2026 6:13:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:13:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:13:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:13:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:13:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:13:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 06:13:43 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786083223, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.133.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:13:43.443291 9750 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.365s user 0.019s sys 0.265s
I20260807 06:13:43.443495 9750 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 06:13:43.443665 9750 mini_ranger_kms.cc:401] 127.9.133.148:40941/kms/v1/keys
Aug 07 06:13:43 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083223, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.133.148@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
Aug 07 06:13:44 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083223, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.133.148@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
Aug 07 06:13:46 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:13:46.683163 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46441
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:46441
--ranger_config_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:40941/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:13:46.840090 25857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:13:46.840453 25857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:13:46.840497 25857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:13:46.847404 25857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:13:46.847564 25857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:13:46.847645 25857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:13:46.847752 25857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:13:46.847798 25857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:13:46.856212 25857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.133.148:40941/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:46441
--ranger_config_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:13:46.857735 25857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:13:46.859189 25857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:13:46.870668 25864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:13:46.870815 25857 server_base.cc:1034] running on GCE node
W20260807 06:13:46.871090 25868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:46.870668 25866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:13:46.871971 25857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:13:46.873461 25857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:13:46.874713 25857 hybrid_clock.cc:648] HybridClock initialized: now 1786083226874684 us; error 51 us; skew 500 ppm
Aug 07 06:13:46 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083226, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:13:46.878177 25857 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:13:46.879405 25857 webserver.cc:460] Webserver started at http://127.9.133.190:45997/ using document root <none> and password file <none>
I20260807 06:13:46.880085 25857 fs_manager.cc:359] Metadata directory not provided
I20260807 06:13:46.880170 25857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:13:46.880473 25857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:13:46 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083226, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:46.953419 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "983fac9cf28848f29579c980067d2325"
format_stamp: "Formatted at 2026-08-07 06:13:46 on dist-test-slave-h3ft"
server_key: "2a786639a71b9bc704c2818756c5adc1"
server_key_iv: "e8e8feebd534826b71e7dd82cc036c7e"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:46.954394 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "983fac9cf28848f29579c980067d2325"
format_stamp: "Formatted at 2026-08-07 06:13:46 on dist-test-slave-h3ft"
server_key: "2a786639a71b9bc704c2818756c5adc1"
server_key_iv: "e8e8feebd534826b71e7dd82cc036c7e"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:46.960284 25857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 06:13:46.983542 25880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:46.985486 25857 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 06:13:46.985664 25857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "983fac9cf28848f29579c980067d2325"
format_stamp: "Formatted at 2026-08-07 06:13:46 on dist-test-slave-h3ft"
server_key: "2a786639a71b9bc704c2818756c5adc1"
server_key_iv: "e8e8feebd534826b71e7dd82cc036c7e"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:46.985841 25857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:13:47.008270 25857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:13:47.012086 25857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:13:47.012336 25857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:13:47.024438 25857 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:46441
I20260807 06:13:47.024477 25958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:46441 every 8 connection(s)
I20260807 06:13:47.025862 25857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:13:47.030568 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:13:47.036005 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 25857
I20260807 06:13:47.036186 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:13:47.036515 9750 external_mini_cluster.cc:1428] Setting key 00524c138d31b1ed2ee8abad7cef87eb
I20260807 06:13:47.040551 25959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325: Bootstrap starting.
I20260807 06:13:47.044180 25959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325: Neither blocks nor log segments found. Creating new log.
I20260807 06:13:47.045271 25959 log.cc:824] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:13:47 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:13:47.048303 25959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325: No bootstrap required, opened a new log
I20260807 06:13:47.053452 25959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983fac9cf28848f29579c980067d2325" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46441 } }
I20260807 06:13:47.053771 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:47.038373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.9.133.190:46441)
0807 06:13:47.039076 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:47.039105 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:47.039152 (+ 47us) server_negotiation.cc:407] Connection header received
0807 06:13:47.039858 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:47.039879 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:47.040292 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:47.040688 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:13:47.041668 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:47.042787 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:47.043766 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:47.044368 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:47.044583 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:47.047958 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:13:47.047964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:13:47.047981 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:13:47.048010 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:13:47.050467 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:47.050957 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:47.050972 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:47.050977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:47.051151 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:47.051460 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:47.051464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:47.051465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:47.051937 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:13:47.052192 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:47.052457 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 06:13:47.052834 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":187,"threads_started":1}
I20260807 06:13:47.053833 25959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:13:47.053897 25959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 983fac9cf28848f29579c980067d2325, State: Initialized, Role: FOLLOWER
I20260807 06:13:47.054518 25959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983fac9cf28848f29579c980067d2325" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46441 } }
I20260807 06:13:47.054720 25959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:13:47.054797 25959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:13:47.054917 25959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:13:47.056325 25959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983fac9cf28848f29579c980067d2325" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46441 } }
I20260807 06:13:47.056751 25959 leader_election.cc:302] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 983fac9cf28848f29579c980067d2325; no voters:
I20260807 06:13:47.057163 25959 leader_election.cc:288] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:13:47.057292 25964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:13:47.057571 25964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [term 1 LEADER]: Becoming Leader. State: Replica: 983fac9cf28848f29579c980067d2325, State: Running, Role: LEADER
I20260807 06:13:47.057973 25964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983fac9cf28848f29579c980067d2325" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46441 } }
I20260807 06:13:47.058729 25959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:13:47.060073 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "983fac9cf28848f29579c980067d2325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983fac9cf28848f29579c980067d2325" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46441 } } }
I20260807 06:13:47.060211 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [sys.catalog]: This master's current role is: LEADER
I20260807 06:13:47.060536 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 983fac9cf28848f29579c980067d2325. Latest consensus state: current_term: 1 leader_uuid: "983fac9cf28848f29579c980067d2325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "983fac9cf28848f29579c980067d2325" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46441 } } }
I20260807 06:13:47.060618 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325 [sys.catalog]: This master's current role is: LEADER
I20260807 06:13:47.060953 25978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:13:47.065238 25978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:13:47.067819 25959 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:13:47.082569 25978 catalog_manager.cc:1347] Generated new cluster ID: 31ded46470c34669857a3dd4e9d5a485
I20260807 06:13:47.082695 25978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:13:47.105959 25978 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:13:47.108147 25978 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:13:47.125367 25978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325: Generated new TSK 0
I20260807 06:13:47.126916 25978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:13:47.987135 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:40941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46441
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:13:48.164786 26019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:13:48.165138 26019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:13:48.165190 26019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:13:48.169654 26019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:13:48.169852 26019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:13:48.170022 26019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:13:48.177098 26019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.133.148:40941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:13:48.178629 26019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:13:48.180521 26019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:13:48.193954 26031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:48.194020 26035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:48.193993 26032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:13:48.194621 26019 server_base.cc:1034] running on GCE node
I20260807 06:13:48.195210 26019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:13:48.196058 26019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:13:48.197299 26019 hybrid_clock.cc:648] HybridClock initialized: now 1786083228197281 us; error 63 us; skew 500 ppm
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:13:48.203768 26019 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:13:48.205056 26019 webserver.cc:460] Webserver started at http://127.9.133.129:45539/ using document root <none> and password file <none>
I20260807 06:13:48.205798 26019 fs_manager.cc:359] Metadata directory not provided
I20260807 06:13:48.205914 26019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:13:48.206236 26019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:48.228608 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "ffedf181f6214e758661ac85d16e77ca"
format_stamp: "Formatted at 2026-08-07 06:13:48 on dist-test-slave-h3ft"
server_key: "32802dad7a1296a27c7b1981beeeb7fc"
server_key_iv: "b446292313401763ba542fa72b3da503"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:48.229921 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "ffedf181f6214e758661ac85d16e77ca"
format_stamp: "Formatted at 2026-08-07 06:13:48 on dist-test-slave-h3ft"
server_key: "32802dad7a1296a27c7b1981beeeb7fc"
server_key_iv: "b446292313401763ba542fa72b3da503"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:48.237478 26019 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.001s
I20260807 06:13:48.260066 26048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:48.262377 26019 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260807 06:13:48.262610 26019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "ffedf181f6214e758661ac85d16e77ca"
format_stamp: "Formatted at 2026-08-07 06:13:48 on dist-test-slave-h3ft"
server_key: "32802dad7a1296a27c7b1981beeeb7fc"
server_key_iv: "b446292313401763ba542fa72b3da503"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:48.262908 26019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:13:48.292060 26019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:13:48.296694 26019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:13:48.297031 26019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:13:48.298092 26019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:13:48.299515 26019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:13:48.299664 26019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 06:13:48.299815 26019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:13:48.299909 26019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:48.316004 26019 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:46801
I20260807 06:13:48.317538 26019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:13:48.318219 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 26019
I20260807 06:13:48.318369 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:13:48.318775 9750 external_mini_cluster.cc:1428] Setting key 18aa07875038bc88565133ab94c49dd6
I20260807 06:13:48.318967 26232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:46801 every 8 connection(s)
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:13:48.342878 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:48.325732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:56021 (local address 127.9.133.190:46441)
0807 06:13:48.328229 (+ 2497us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:48.328234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:48.328251 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:13:48.328302 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:48.328305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:48.328361 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:48.328464 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:13:48.330534 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.331600 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.333576 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.333971 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.334053 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:48.337888 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:13:48.337893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:13:48.337897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:13:48.337968 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:13:48.340157 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:48.340932 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:48.340935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:48.340937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:48.341028 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:48.341432 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:48.341436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:48.341438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:48.341667 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:13:48.341782 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:48.342479 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 06:13:48.342648 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2398,"thread_start_us":90,"threads_started":1}
I20260807 06:13:48.345257 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:48.324134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46441 (local address 127.9.133.129:56021)
0807 06:13:48.326250 (+ 2116us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:48.326318 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:48.327675 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:48.328848 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:48.328858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:48.329414 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:13:48.330322 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:48.330357 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.331772 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.331775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:48.333430 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:48.333442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.334304 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.334308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:48.334535 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:48.335468 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:13:48.335497 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:13:48.337611 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:13:48.340307 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:13:48.340314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:13:48.340337 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:13:48.340745 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:13:48.341131 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:13:48.341135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:13:48.341137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:13:48.341313 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:13:48.341795 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:13:48.341804 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:13:48.342143 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 06:13:48.343141 (+ 998us) client_negotiation.cc:240] Negotiation successful
0807 06:13:48.343636 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1834,"thread_start_us":341,"threads_started":1}
I20260807 06:13:48.347523 26233 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46441
I20260807 06:13:48.347869 26233 heartbeater.cc:464] Registering TS with master...
I20260807 06:13:48.348698 26233 heartbeater.cc:511] Master 127.9.133.190:46441 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:13:48.351651 25906 ts_manager.cc:194] Registered new tserver with Master: ffedf181f6214e758661ac85d16e77ca (127.9.133.129:46801)
I20260807 06:13:48.353633 25906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:56021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:13:48.379487 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:40941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46441
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 06:13:48.541033 26244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:13:48.541517 26244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:13:48.541649 26244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:13:48.546298 26244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:13:48.546411 26244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:13:48.546523 26244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:13:48.554008 26244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.133.148:40941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:13:48.555959 26244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:13:48.557340 26244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:13:48.571221 26250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:48.571213 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:48.571617 26254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:13:48.572703 26244 server_base.cc:1034] running on GCE node
I20260807 06:13:48.573294 26244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:13:48.574258 26244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:13:48.575548 26244 hybrid_clock.cc:648] HybridClock initialized: now 1786083228575465 us; error 108 us; skew 500 ppm
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:13:48.580133 26244 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:13:48.581607 26244 webserver.cc:460] Webserver started at http://127.9.133.130:37463/ using document root <none> and password file <none>
I20260807 06:13:48.582585 26244 fs_manager.cc:359] Metadata directory not provided
I20260807 06:13:48.582697 26244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:13:48.583130 26244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:48.609269 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "a35451d8657842b4bfdf4ab49bae83b6"
format_stamp: "Formatted at 2026-08-07 06:13:48 on dist-test-slave-h3ft"
server_key: "f973f46d63731495eeaf837077c7047b"
server_key_iv: "37839b8c3ebc44a696a20868bad36b4e"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:48.610314 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "a35451d8657842b4bfdf4ab49bae83b6"
format_stamp: "Formatted at 2026-08-07 06:13:48 on dist-test-slave-h3ft"
server_key: "f973f46d63731495eeaf837077c7047b"
server_key_iv: "37839b8c3ebc44a696a20868bad36b4e"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:48.619079 26244 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.001s
I20260807 06:13:48.653752 26273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:48.656189 26244 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 06:13:48.656374 26244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "a35451d8657842b4bfdf4ab49bae83b6"
format_stamp: "Formatted at 2026-08-07 06:13:48 on dist-test-slave-h3ft"
server_key: "f973f46d63731495eeaf837077c7047b"
server_key_iv: "37839b8c3ebc44a696a20868bad36b4e"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:48.656580 26244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:13:48.703233 26244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:13:48.708586 26244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:13:48.708925 26244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:13:48.710032 26244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:13:48.711627 26244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:13:48.711727 26244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:48.711809 26244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:13:48.711856 26244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:48.729197 26244 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:38829
I20260807 06:13:48.729260 26437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:38829 every 8 connection(s)
I20260807 06:13:48.730857 26244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:13:48.735147 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 26244
I20260807 06:13:48.735321 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:13:48.735792 9750 external_mini_cluster.cc:1428] Setting key d359de4749593ebfc485a95a5ded2e51
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Aug 07 06:13:48 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:13:48.769163 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:48.734431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:45967 (local address 127.9.133.190:46441)
0807 06:13:48.734653 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:48.734658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:48.743309 (+ 8651us) server_negotiation.cc:407] Connection header received
0807 06:13:48.749258 (+ 5949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:48.749265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:48.749409 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:48.749492 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:13:48.752393 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.753348 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.755065 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.755463 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.755593 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:48.761641 (+ 6048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:13:48.761647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:13:48.761654 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:13:48.761696 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:13:48.763596 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:48.766702 (+ 3106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:48.766705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:48.766706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:48.766806 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:48.767310 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:48.767313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:48.767315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:48.767565 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:13:48.767688 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:48.768684 (+ 996us) server_negotiation.cc:299] Negotiation successful
0807 06:13:48.768843 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 06:13:48.770256 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:48.734746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46441 (local address 127.9.133.130:45967)
0807 06:13:48.742970 (+ 8224us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:48.743115 (+ 145us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:48.746840 (+ 3725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:48.750294 (+ 3454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:48.750309 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:48.750836 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:13:48.752093 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:48.752128 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.753558 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.753561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:48.754865 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:48.754880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:48.755703 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:48.755709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:48.756042 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:48.756965 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:13:48.756992 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:13:48.761382 (+ 4390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:13:48.766091 (+ 4709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:13:48.766099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:13:48.766122 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:13:48.766559 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:13:48.766947 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:13:48.766951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:13:48.766953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:13:48.767192 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:13:48.767686 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:13:48.767693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:13:48.768046 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 06:13:48.768275 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 06:13:48.768737 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7764,"thread_start_us":222,"threads_started":1}
I20260807 06:13:48.771786 26439 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46441
I20260807 06:13:48.772105 26439 heartbeater.cc:464] Registering TS with master...
I20260807 06:13:48.772773 26439 heartbeater.cc:511] Master 127.9.133.190:46441 requested a full tablet report, sending...
I20260807 06:13:48.774199 25906 ts_manager.cc:194] Registered new tserver with Master: a35451d8657842b4bfdf4ab49bae83b6 (127.9.133.130:38829)
I20260807 06:13:48.774883 25906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:45967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:13:48.783130 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:40941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46441
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:13:49.000564 26451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:13:49.001099 26451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:13:49.001187 26451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:13:49.006326 26451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:13:49.006505 26451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:13:49.006644 26451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:13:49.013754 26451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.133.148:40941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
Aug 07 06:13:49 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:49.015753 26451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:13:49.017302 26451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:13:49.035511 26461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:49.035992 26462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:13:49.036334 26464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:13:49.041265 26451 server_base.cc:1034] running on GCE node
I20260807 06:13:49.042120 26451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:13:49.043270 26451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:13:49.044598 26451 hybrid_clock.cc:648] HybridClock initialized: now 1786083229044536 us; error 63 us; skew 500 ppm
Aug 07 06:13:49 dist-test-slave-h3ft krb5kdc[25174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:13:49.049418 26451 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:13:49.050938 26451 webserver.cc:460] Webserver started at http://127.9.133.131:40431/ using document root <none> and password file <none>
I20260807 06:13:49.051944 26451 fs_manager.cc:359] Metadata directory not provided
I20260807 06:13:49.052044 26451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:13:49.052332 26451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:13:49 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:49.078044 26451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "fdf6e54672f441f79cd484ed4515b68d"
format_stamp: "Formatted at 2026-08-07 06:13:49 on dist-test-slave-h3ft"
server_key: "d721f1a3e77142c626f1d6d13e299a2f"
server_key_iv: "af104ad0e1b6f3998da28166e695d36a"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:49.079089 26451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "fdf6e54672f441f79cd484ed4515b68d"
format_stamp: "Formatted at 2026-08-07 06:13:49 on dist-test-slave-h3ft"
server_key: "d721f1a3e77142c626f1d6d13e299a2f"
server_key_iv: "af104ad0e1b6f3998da28166e695d36a"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:49.095000 26451 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.005s sys 0.000s
I20260807 06:13:49.145651 26472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:49.147974 26451 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 06:13:49.148145 26451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "fdf6e54672f441f79cd484ed4515b68d"
format_stamp: "Formatted at 2026-08-07 06:13:49 on dist-test-slave-h3ft"
server_key: "d721f1a3e77142c626f1d6d13e299a2f"
server_key_iv: "af104ad0e1b6f3998da28166e695d36a"
server_key_version: "kuduclusterkey@0"
I20260807 06:13:49.148322 26451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:13:49.169298 26451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:13:49.173820 26451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:13:49.174152 26451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:13:49.175029 26451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:13:49.176381 26451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:13:49.176488 26451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 06:13:49.176570 26451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:13:49.176622 26451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:13:49.193329 26451 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:41795
I20260807 06:13:49.193692 26659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:41795 every 8 connection(s)
I20260807 06:13:49.194775 26451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:13:49.204761 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 26451
I20260807 06:13:49.204926 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:13:49.205288 9750 external_mini_cluster.cc:1428] Setting key fd0bdb89cd5b68ec0cdbfcfb1403b005
Aug 07 06:13:49 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:13:49.215946 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.199985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:41677 (local address 127.9.133.190:46441)
0807 06:13:49.200137 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.200142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.200163 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:13:49.201201 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.201206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.201278 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.201381 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:13:49.203114 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.204022 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.206181 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.206524 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.206624 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.210817 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:13:49.210823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:13:49.210826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:13:49.210870 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:13:49.213061 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:49.213775 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:49.213780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:49.213782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:49.213902 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:49.214368 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:49.214372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:49.214374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:49.214642 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:13:49.214754 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.215552 (+ 798us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.215714 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:13:49.217255 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.198705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46441 (local address 127.9.133.131:41677)
0807 06:13:49.199483 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.199556 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:49.200813 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:49.201612 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:49.201623 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:49.202076 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:13:49.202888 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.202911 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.204190 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.204194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.206002 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.206017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.206898 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.206903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.207191 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.208177 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:13:49.208205 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:13:49.210606 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:13:49.213202 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:13:49.213209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:13:49.213232 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:13:49.213633 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:13:49.214019 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:13:49.214022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:13:49.214024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:13:49.214242 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:13:49.214779 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:13:49.214787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:13:49.215214 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 06:13:49.215457 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 06:13:49.215863 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":184,"threads_started":1}
I20260807 06:13:49.219108 26661 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46441
I20260807 06:13:49.219338 26661 heartbeater.cc:464] Registering TS with master...
I20260807 06:13:49.220080 26661 heartbeater.cc:511] Master 127.9.133.190:46441 requested a full tablet report, sending...
I20260807 06:13:49.221740 25906 ts_manager.cc:194] Registered new tserver with Master: fdf6e54672f441f79cd484ed4515b68d (127.9.133.131:41795)
I20260807 06:13:49.222637 25906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:41677
I20260807 06:13:49.232856 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:13:49.247088 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.236144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.9.133.190:46441)
0807 06:13:49.236486 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.236492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.236604 (+ 112us) server_negotiation.cc:407] Connection header received
0807 06:13:49.236761 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.236765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.236843 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.236944 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:13:49.238034 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.238918 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.240271 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.240670 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.240801 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.242726 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:13:49.242731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:13:49.242735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:13:49.242766 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:13:49.244850 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:49.245394 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:49.245399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:49.245401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:49.245516 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:13:49.245882 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:13:49.245886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:13:49.245888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:13:49.246158 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:13:49.246323 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.246653 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.246797 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260807 06:13:49.251942 25906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:13:49.258175 25906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:13:49.285784 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.274082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:41795 (local address 127.0.0.1:57506)
0807 06:13:49.279135 (+ 5053us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.279152 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:49.279273 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:49.280040 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:49.280048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:49.280090 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:13:49.280388 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.280400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.281920 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.281923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.283536 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.283548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.285043 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.285048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.285193 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.285266 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 06:13:49.285401 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 06:13:49.285552 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4986,"mutex_wait_us":166,"spinlock_wait_cycles":6656,"thread_start_us":130,"threads_started":1}
I20260807 06:13:49.286401 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.274655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.9.133.131:41795)
0807 06:13:49.275436 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.275503 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.279402 (+ 3899us) server_negotiation.cc:407] Connection header received
0807 06:13:49.279494 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.279499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.279655 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.279814 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:13:49.280559 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.281777 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.283723 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.284630 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.284747 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.284964 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.286057 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.286187 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":199,"threads_started":1}
I20260807 06:13:49.286523 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.274553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.9.133.129:46801)
0807 06:13:49.274990 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.275057 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.279398 (+ 4341us) server_negotiation.cc:407] Connection header received
0807 06:13:49.279508 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.279514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.279750 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.279928 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:13:49.280542 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.281918 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.284310 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.285295 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.285436 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.285597 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.286220 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.286333 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":146,"threads_started":1}
I20260807 06:13:49.286101 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.272897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:46801 (local address 127.0.0.1:36702)
0807 06:13:49.279122 (+ 6225us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.279139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:49.279256 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:49.280132 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:49.280136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:49.280156 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:13:49.280368 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.280379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.282081 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.282086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.284147 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.284159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.285674 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.285677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.285771 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.285797 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:13:49.285877 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 06:13:49.285947 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6146,"mutex_wait_us":601,"thread_start_us":118,"threads_started":1}
I20260807 06:13:49.287529 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.272799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38829 (local address 127.0.0.1:33312)
0807 06:13:49.278469 (+ 5670us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.278543 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:49.278713 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:49.280040 (+ 1327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:49.280048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:49.280092 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:13:49.280422 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.280432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.282896 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.282900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.284066 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.284077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.287047 (+ 2970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.287051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.287210 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.287239 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:13:49.287314 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 06:13:49.287383 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5435,"mutex_wait_us":113,"spinlock_wait_cycles":8832,"thread_start_us":133,"threads_started":1}
I20260807 06:13:49.288370 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.274027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33312 (local address 127.9.133.130:38829)
0807 06:13:49.279196 (+ 5169us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.279207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.279242 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:13:49.279350 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.279358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.279631 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.279815 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:13:49.280568 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.282737 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.284218 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.285171 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.285285 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.288034 (+ 2749us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.288092 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.288202 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5024,"thread_start_us":196,"threads_started":1}
I20260807 06:13:49.290589 26557 tablet_service.cc:1467] Processing CreateTablet for tablet bbf70dcd6f2649ed966c3a7cfebe37e1 (DEFAULT_TABLE table=test-table [id=65c0ef68e66146fa90c8fa787a29ab7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:13:49.290591 26120 tablet_service.cc:1467] Processing CreateTablet for tablet bbf70dcd6f2649ed966c3a7cfebe37e1 (DEFAULT_TABLE table=test-table [id=65c0ef68e66146fa90c8fa787a29ab7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:13:49.292143 26120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbf70dcd6f2649ed966c3a7cfebe37e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:13:49.292157 26557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbf70dcd6f2649ed966c3a7cfebe37e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:13:49.297560 26331 tablet_service.cc:1467] Processing CreateTablet for tablet bbf70dcd6f2649ed966c3a7cfebe37e1 (DEFAULT_TABLE table=test-table [id=65c0ef68e66146fa90c8fa787a29ab7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:13:49.298861 26331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbf70dcd6f2649ed966c3a7cfebe37e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:13:49.302495 26693 tablet_bootstrap.cc:492] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Bootstrap starting.
I20260807 06:13:49.306247 26693 tablet_bootstrap.cc:654] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Neither blocks nor log segments found. Creating new log.
I20260807 06:13:49.307425 26693 log.cc:824] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:13:49.308044 26694 tablet_bootstrap.cc:492] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Bootstrap starting.
I20260807 06:13:49.310021 26693 tablet_bootstrap.cc:492] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: No bootstrap required, opened a new log
I20260807 06:13:49.310256 26693 ts_tablet_manager.cc:1397] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:13:49.311951 26694 tablet_bootstrap.cc:654] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Neither blocks nor log segments found. Creating new log.
I20260807 06:13:49.313203 26694 log.cc:824] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:13:49.313622 26692 tablet_bootstrap.cc:492] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca: Bootstrap starting.
I20260807 06:13:49.315485 26693 raft_consensus.cc:348] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.315954 26693 raft_consensus.cc:374] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:13:49.316073 26693 raft_consensus.cc:729] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf6e54672f441f79cd484ed4515b68d, State: Initialized, Role: FOLLOWER
I20260807 06:13:49.316140 26694 tablet_bootstrap.cc:492] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: No bootstrap required, opened a new log
I20260807 06:13:49.316378 26694 ts_tablet_manager.cc:1397] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 06:13:49.316709 26692 tablet_bootstrap.cc:654] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca: Neither blocks nor log segments found. Creating new log.
I20260807 06:13:49.316767 26693 consensus_queue.cc:260] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.317864 26692 log.cc:824] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca: Log is configured to *not* fsync() on all Append() calls
I20260807 06:13:49.318045 26693 ts_tablet_manager.cc:1428] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:13:49.318514 26661 heartbeater.cc:503] Master 127.9.133.190:46441 was elected leader, sending a full tablet report...
I20260807 06:13:49.324019 26694 raft_consensus.cc:348] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.324437 26694 raft_consensus.cc:374] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:13:49.324535 26694 raft_consensus.cc:729] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35451d8657842b4bfdf4ab49bae83b6, State: Initialized, Role: FOLLOWER
I20260807 06:13:49.325168 26694 consensus_queue.cc:260] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.326550 26694 ts_tablet_manager.cc:1428] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260807 06:13:49.327155 26439 heartbeater.cc:503] Master 127.9.133.190:46441 was elected leader, sending a full tablet report...
I20260807 06:13:49.331688 26692 tablet_bootstrap.cc:492] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca: No bootstrap required, opened a new log
I20260807 06:13:49.332005 26692 ts_tablet_manager.cc:1397] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260807 06:13:49.337229 26692 raft_consensus.cc:348] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.337641 26692 raft_consensus.cc:374] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:13:49.337756 26692 raft_consensus.cc:729] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffedf181f6214e758661ac85d16e77ca, State: Initialized, Role: FOLLOWER
I20260807 06:13:49.338384 26692 consensus_queue.cc:260] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.339357 26233 heartbeater.cc:503] Master 127.9.133.190:46441 was elected leader, sending a full tablet report...
I20260807 06:13:49.339874 26692 ts_tablet_manager.cc:1428] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
Aug 07 06:13:49 dist-test-slave-h3ft krb5kdc[25174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083228, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:13:49.446141 26697 raft_consensus.cc:482] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:13:49.446607 26697 raft_consensus.cc:504] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.448601 26697 leader_election.cc:288] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a35451d8657842b4bfdf4ab49bae83b6 (127.9.133.130:38829), ffedf181f6214e758661ac85d16e77ca (127.9.133.129:46801)
W20260807 06:13:49.450977 26663 tablet.cc:2369] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:13:49.455233 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.448849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38829 (local address 127.9.133.131:43697)
0807 06:13:49.449025 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.449045 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:49.449165 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:49.450466 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:49.450469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:49.450544 (+ 75us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:13:49.450754 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.450762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.452012 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.452015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.453422 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.453435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.454575 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.454578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.454767 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.454862 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 06:13:49.454939 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 06:13:49.455063 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":163,"spinlock_wait_cycles":24832}
I20260807 06:13:49.455233 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.449927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:51069 (local address 127.9.133.129:46801)
0807 06:13:49.450083 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.450089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.450104 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:13:49.450178 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.450181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.450237 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.450318 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:13:49.450842 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.451742 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.453620 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.454266 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.454372 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.454442 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.455020 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.455105 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:13:49.455721 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.449048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:46801 (local address 127.9.133.131:51069)
0807 06:13:49.449534 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.449550 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:13:49.449740 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:13:49.450466 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:13:49.450469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:13:49.450490 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:13:49.450658 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.450666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.451904 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.451907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.453487 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:13:49.453494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.454585 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.454587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:13:49.454786 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.454870 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 06:13:49.455399 (+ 529us) client_negotiation.cc:240] Negotiation successful
0807 06:13:49.455504 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":159,"thread_start_us":248,"threads_started":1}
I20260807 06:13:49.456565 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.449730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:43697 (local address 127.9.133.130:38829)
0807 06:13:49.450117 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.450123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.450139 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:13:49.450196 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.450199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.450249 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.450320 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:13:49.450902 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.451742 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.453618 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.454262 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.454376 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.454443 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.456312 (+ 1869us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.456412 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260807 06:13:49.457115 26129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbf70dcd6f2649ed966c3a7cfebe37e1" candidate_uuid: "fdf6e54672f441f79cd484ed4515b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffedf181f6214e758661ac85d16e77ca" is_pre_election: true
I20260807 06:13:49.457597 26129 raft_consensus.cc:2454] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdf6e54672f441f79cd484ed4515b68d in term 0.
I20260807 06:13:49.457788 26368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbf70dcd6f2649ed966c3a7cfebe37e1" candidate_uuid: "fdf6e54672f441f79cd484ed4515b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35451d8657842b4bfdf4ab49bae83b6" is_pre_election: true
I20260807 06:13:49.458132 26368 raft_consensus.cc:2454] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdf6e54672f441f79cd484ed4515b68d in term 0.
I20260807 06:13:49.458238 26479 leader_election.cc:302] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fdf6e54672f441f79cd484ed4515b68d, ffedf181f6214e758661ac85d16e77ca; no voters:
I20260807 06:13:49.458987 26697 raft_consensus.cc:2781] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:13:49.459149 26697 raft_consensus.cc:482] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:13:49.459204 26697 raft_consensus.cc:3037] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:13:49.461347 26697 raft_consensus.cc:504] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.462288 26368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbf70dcd6f2649ed966c3a7cfebe37e1" candidate_uuid: "fdf6e54672f441f79cd484ed4515b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35451d8657842b4bfdf4ab49bae83b6"
I20260807 06:13:49.462287 26129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbf70dcd6f2649ed966c3a7cfebe37e1" candidate_uuid: "fdf6e54672f441f79cd484ed4515b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffedf181f6214e758661ac85d16e77ca"
I20260807 06:13:49.462445 26129 raft_consensus.cc:3037] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:13:49.462432 26368 raft_consensus.cc:3037] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:13:49.463882 26697 leader_election.cc:288] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [CANDIDATE]: Term 1 election: Requested vote from peers a35451d8657842b4bfdf4ab49bae83b6 (127.9.133.130:38829), ffedf181f6214e758661ac85d16e77ca (127.9.133.129:46801)
I20260807 06:13:49.464123 26368 raft_consensus.cc:2454] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdf6e54672f441f79cd484ed4515b68d in term 1.
I20260807 06:13:49.464123 26129 raft_consensus.cc:2454] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdf6e54672f441f79cd484ed4515b68d in term 1.
I20260807 06:13:49.464483 26476 leader_election.cc:302] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a35451d8657842b4bfdf4ab49bae83b6, fdf6e54672f441f79cd484ed4515b68d; no voters:
I20260807 06:13:49.464746 26697 raft_consensus.cc:2781] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:13:49.465093 26697 raft_consensus.cc:686] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 1 LEADER]: Becoming Leader. State: Replica: fdf6e54672f441f79cd484ed4515b68d, State: Running, Role: LEADER
I20260807 06:13:49.465610 26697 consensus_queue.cc:237] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } }
I20260807 06:13:49.470307 25905 catalog_manager.cc:5495] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d reported cstate change: term changed from 0 to 1, leader changed from <none> to fdf6e54672f441f79cd484ed4515b68d (127.9.133.131). New cstate: current_term: 1 leader_uuid: "fdf6e54672f441f79cd484ed4515b68d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdf6e54672f441f79cd484ed4515b68d" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41795 } health_report { overall_health: HEALTHY } } }
W20260807 06:13:49.483137 26440 tablet.cc:2369] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:13:49.487977 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.482604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.9.133.131:41795)
0807 06:13:49.482791 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:13:49.482796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:13:49.482817 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:13:49.482924 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:13:49.482928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:13:49.483070 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:13:49.483187 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:13:49.483537 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.484119 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.485508 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:13:49.485861 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:13:49.485965 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:13:49.486481 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:13:49.487155 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:13:49.487278 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 06:13:49.487742 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 06:13:49.487811 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:13:49.498140 26129 raft_consensus.cc:1261] T bbf70dcd6f2649ed966c3a7cfebe37e1 P ffedf181f6214e758661ac85d16e77ca [term 1 FOLLOWER]: Refusing update from remote peer fdf6e54672f441f79cd484ed4515b68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:13:49.498493 26368 raft_consensus.cc:1261] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 1 FOLLOWER]: Refusing update from remote peer fdf6e54672f441f79cd484ed4515b68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:13:49.499428 26716 consensus_queue.cc:1035] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffedf181f6214e758661ac85d16e77ca" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 46801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:13:49.500000 26716 consensus_queue.cc:1035] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a35451d8657842b4bfdf4ab49bae83b6" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:13:49.518035 26724 mvcc.cc:205] Tried to move back new op lower bound from 7315796908011204608 to 7315796907894714368. Current Snapshot: MvccSnapshot[applied={T|T < 7315796908011204608}]
I20260807 06:13:49.520697 25906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:13:49.520928 25906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:13:49.524183 25906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 983fac9cf28848f29579c980067d2325: Sending DeleteTablet for 3 replicas of tablet bbf70dcd6f2649ed966c3a7cfebe37e1
I20260807 06:13:49.525466 26120 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf70dcd6f2649ed966c3a7cfebe37e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:13:49 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:36702
I20260807 06:13:49.526885 26723 mvcc.cc:205] Tried to move back new op lower bound from 7315796908011204608 to 7315796907894714368. Current Snapshot: MvccSnapshot[applied={T|T < 7315796908011204608}]
I20260807 06:13:49.527112 26557 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf70dcd6f2649ed966c3a7cfebe37e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:13:49 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:57506
I20260807 06:13:49.527189 26331 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf70dcd6f2649ed966c3a7cfebe37e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:13:49 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:33312
I20260807 06:13:49.530622 26738 tablet_replica.cc:330] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: stopping tablet replica
I20260807 06:13:49.531279 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 26019
I20260807 06:13:49.559450 26738 raft_consensus.cc:2229] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:13:49.560185 26738 raft_consensus.cc:2258] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:13:49.562297 26737 tablet_replica.cc:330] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: stopping tablet replica
I20260807 06:13:49.562928 26737 raft_consensus.cc:2229] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:13:49.564949 26738 ts_tablet_manager.cc:1905] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:13:49.567427 26737 raft_consensus.cc:2258] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:13:49.569535 26737 ts_tablet_manager.cc:1905] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:13:49.570168 26738 ts_tablet_manager.cc:1918] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:13:49.570297 26738 log.cc:1196] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/bbf70dcd6f2649ed966c3a7cfebe37e1
I20260807 06:13:49.570670 26738 ts_tablet_manager.cc:1939] T bbf70dcd6f2649ed966c3a7cfebe37e1 P a35451d8657842b4bfdf4ab49bae83b6: Deleting consensus metadata
I20260807 06:13:49.572384 25884 catalog_manager.cc:4841] TS a35451d8657842b4bfdf4ab49bae83b6 (127.9.133.130:38829): tablet bbf70dcd6f2649ed966c3a7cfebe37e1 (table test-table [id=65c0ef68e66146fa90c8fa787a29ab7b]) successfully deleted
I20260807 06:13:49.573354 26737 ts_tablet_manager.cc:1918] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:13:49.573547 26737 log.cc:1196] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/bbf70dcd6f2649ed966c3a7cfebe37e1
I20260807 06:13:49.573995 26737 ts_tablet_manager.cc:1939] T bbf70dcd6f2649ed966c3a7cfebe37e1 P fdf6e54672f441f79cd484ed4515b68d: Deleting consensus metadata
W20260807 06:13:49.577400 25889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:46801 (error 108)
I20260807 06:13:49.578236 25884 catalog_manager.cc:4841] TS fdf6e54672f441f79cd484ed4515b68d (127.9.133.131:41795): tablet bbf70dcd6f2649ed966c3a7cfebe37e1 (table test-table [id=65c0ef68e66146fa90c8fa787a29ab7b]) successfully deleted
I20260807 06:13:49.579027 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.578629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:46801 (local address 127.0.0.1:36710)
0807 06:13:49.578803 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.578891 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 06:13:49.579185 25889 catalog_manager.cc:4584] TS ffedf181f6214e758661ac85d16e77ca (127.9.133.129:46801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbf70dcd6f2649ed966c3a7cfebe37e1: Network error: Client connection negotiation failed: client connection to 127.9.133.129:46801: connect: Connection refused (error 111)
I20260807 06:13:49.583079 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 26244
I20260807 06:13:49.606336 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 26451
I20260807 06:13:49.642596 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.642059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:46801 (local address 127.0.0.1:36718)
0807 06:13:49.642363 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.642447 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260807 06:13:49.643786 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:13:49.643459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:46801 (local address 127.0.0.1:36732)
0807 06:13:49.643619 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 06:13:49.643693 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 06:13:49.653554 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 25857
I20260807 06:13:49.671847 9750 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 06:13:50.032487 9750 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 06:13:50.032769 9750 mini_ranger.cc:66] Stopping Ranger...
I20260807 06:13:50.389832 9750 mini_ranger.cc:68] Stopped Ranger
2026-08-07 06:13:50.390 UTC [25224] LOG: received smart shutdown request
2026-08-07 06:13:50.402 UTC [25224] LOG: background worker "logical replication launcher" (PID 25238) exited with exit code 1
2026-08-07 06:13:50.402 UTC [25230] LOG: shutting down
2026-08-07 06:13:50.561 UTC [25224] LOG: database system is shut down
2026-08-07T06:13:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54090 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:13:50 dist-test-slave-h3ft krb5kdc[26754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:13:50 dist-test-slave-h3ft krb5kdc[26754](info): set up 2 sockets
Aug 07 06:13:50 dist-test-slave-h3ft krb5kdc[26754](info): commencing operation
krb5kdc: starting...
W20260807 06:13:51.973604 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:13:52 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:13:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:13:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/rangeradmin_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/rangeradmin_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/rangerlookup_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/rangerlookup_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
I20260807 06:13:52.188131 9750 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrsqWAA/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/postgres -l logfile start
2026-08-07 06:13:55.027 UTC [26794] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 06:13:55.028 UTC [26794] LOG: listening on IPv4 address "127.9.133.148", port 53545
2026-08-07 06:13:55.031 UTC [26794] LOG: listening on Unix socket "/tmp/.s.PGSQL.53545"
2026-08-07 06:13:55.072 UTC [26797] LOG: database system was shut down at 2026-08-07 06:13:53 UTC
2026-08-07 06:13:55.078 UTC [26794] LOG: database system is ready to accept connections
I20260807 06:13:56.230767 9750 mini_postgres.cc:96] Postgres bound to 53545
2026-08-07 06:13:56.240 UTC [26808] FATAL: database "slave" does not exist
127.9.133.148:53545 - accepting connections
I20260807 06:13:56.241613 9750 mini_ranger.cc:161] Starting Ranger...
I20260807 06:13:56.261209 9750 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 06:13:56.796739 9750 mini_ranger.cc:87] Created ranger Postgres database
I20260807 06:13:56.796867 9750 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-admin
2026-08-07 06:13:57,126 [I] DB FLAVOR :POSTGRES
2026-08-07 06:13:57,127 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:13:57,127 [I] Checking connection..
2026-08-07 06:13:57,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 06:13:57,430 [I] Checking connection passed.
2026-08-07 06:13:57,431 [I] --------- Verifying version history table ---------
2026-08-07 06:13:57,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:13:57,740 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:13:57,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:13:58,041 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 06:13:58,041 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 06:13:58,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 06:13:58.329 UTC [26905] WARNING: there is no transaction in progress
2026-08-07 06:13:58,339 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 06:13:58,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 06:13:58,631 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 06:13:58,631 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:13:58,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 06:13:58,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 06:13:59,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-07 06:13:59,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:13:59,760 [I] Table x_portal_user does not exist in database ranger
2026-08-07 06:13:59,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:14:00,053 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 06:14:00,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:14:00,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:14:00,626 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 06:14:00,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 06:14:01.729 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.752 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.772 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.791 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.811 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.835 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.908 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.918 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.939 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.953 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.967 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:01.987 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:02.001 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:02.010 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:02.019 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:02.029 UTC [27052] WARNING: there is no transaction in progress
2026-08-07 06:14:02,933 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 06:14:02,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 06:14:03,256 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 06:14:03,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 06:14:03,556 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 06:14:03,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 06:14:03,892 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 06:14:03,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 06:14:04,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 06:14:04,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-07 06:14:04,817 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 06:14:04,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 06:14:05,116 [I] DB_PATCHES have already been applied
I20260807 06:14:05.124467 9750 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrsqWAA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 06:14:05.124588 9750 mini_ranger.cc:193] Using host: 127.9.133.148
I20260807 06:14:05.131238 9750 mini_ranger.cc:227] Ranger admin URL: http://127.9.133.148:52771
Aug 07, 2026 6:14:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:14:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:14:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:14:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:14:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:14:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:14:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 06:14:06.595459 9750 mini_ranger.cc:160] Time spent starting Ranger: real 10.354s user 0.000s sys 0.017s
Aug 07, 2026 6:14:33 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:14:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:14:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:14:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:14:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 06:14:35.589673 9750 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.133.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.133.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/rangerkms_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.133.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/rangerkms_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.133.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.133.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.133.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/HTTP_127.9.133.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 06:14:35 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083275, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 06:14:35.728542 9750 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 06:14:35.750635 9750 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 06:14:36.579948 9750 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 06:14:36.580108 9750 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms
I20260807 06:14:36.580155 9750 mini_ranger_kms.cc:227] Using postgres at 127.9.133.148:53545
2026-08-07 06:14:37,075 [I] DB FLAVOR :POSTGRES
2026-08-07 06:14:37,075 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 06:14:37,075 [I] Checking connection
2026-08-07 06:14:37,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 06:14:37,503 [I] connection success
2026-08-07 06:14:37,504 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 06:14:37,504 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 06:14:37,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 06:14:37,823 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 06:14:37,824 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 06:14:37,824 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 06:14:37,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.133.148:53545/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 06:14:38,218 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 06:14:38.882764 9750 mini_ranger_kms.cc:310] Created kms service
I20260807 06:14:39.170469 9750 mini_ranger_kms.cc:326] Created kudu user
I20260807 06:14:39.224346 9750 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 06:14:39 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083275, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:39.740834 9750 mini_ranger_kms.cc:384] Added ranger policy
I20260807 06:14:39.741065 9750 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrsqWAA/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrsqWAA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 06:14:39.741099 9750 mini_ranger_kms.cc:240] Using host: 127.9.133.148
I20260807 06:14:39.748885 9750 mini_ranger_kms.cc:276] Ranger KMS PID: 27329
I20260807 06:14:39.749009 9750 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.133.148:36951
Aug 07, 2026 6:14:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:14:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 06:14:42 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786083282, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.133.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:42.576779 9750 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.848s user 0.035s sys 0.409s
I20260807 06:14:42.577090 9750 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 06:14:42.577203 9750 mini_ranger_kms.cc:401] 127.9.133.148:36951/kms/v1/keys
Aug 07 06:14:42 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083282, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.133.148@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
Aug 07 06:14:43 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083282, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.133.148@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
Aug 07 06:14:46 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:14:46.211488 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46395
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:46395
--ranger_config_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:46.362775 27400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:46.363199 27400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:46.363281 27400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:46.367498 27400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:46.367614 27400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:46.367648 27400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:46.367705 27400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:46.367759 27400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:46.374025 27400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.133.148:36951/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:46395
--ranger_config_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:46.375538 27400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:46.376838 27400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:46.387919 27410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:46.387844 27415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:46.387915 27417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:46.388154 27400 server_base.cc:1034] running on GCE node
I20260807 06:14:46.388927 27400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:46.390309 27400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:46.391496 27400 hybrid_clock.cc:648] HybridClock initialized: now 1786083286391475 us; error 34 us; skew 500 ppm
Aug 07 06:14:46 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:46.399667 27400 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:14:46.400802 27400 webserver.cc:460] Webserver started at http://127.9.133.190:43503/ using document root <none> and password file <none>
I20260807 06:14:46.401484 27400 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:46.401546 27400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:46.401762 27400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:14:46 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:46.469661 27400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "c2cf50b69326447983d59947c5f7f409"
format_stamp: "Formatted at 2026-08-07 06:14:46 on dist-test-slave-h3ft"
server_key: "b9071416305528a193822e9f17583832"
server_key_iv: "670e5af1f4814bfec78c407135bb52e7"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:46.470655 27400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "c2cf50b69326447983d59947c5f7f409"
format_stamp: "Formatted at 2026-08-07 06:14:46 on dist-test-slave-h3ft"
server_key: "b9071416305528a193822e9f17583832"
server_key_iv: "670e5af1f4814bfec78c407135bb52e7"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:46.476928 27400 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 06:14:46.498643 27427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:46.500671 27400 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 06:14:46.500885 27400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "c2cf50b69326447983d59947c5f7f409"
format_stamp: "Formatted at 2026-08-07 06:14:46 on dist-test-slave-h3ft"
server_key: "b9071416305528a193822e9f17583832"
server_key_iv: "670e5af1f4814bfec78c407135bb52e7"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:46.501072 27400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:46.522392 27400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:46.526247 27400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:46.526515 27400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:46.537880 27400 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:46395
I20260807 06:14:46.538048 27506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:46395 every 8 connection(s)
I20260807 06:14:46.539263 27400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:14:46.542634 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 27400
I20260807 06:14:46.542833 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:14:46.543308 9750 external_mini_cluster.cc:1428] Setting key 932d3e3c1a7f028bb9a804b53d721218
I20260807 06:14:46.543426 27507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:46.553634 27507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Bootstrap starting.
Aug 07 06:14:46 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:46.557672 27507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:46.559471 27507 log.cc:824] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:46.562322 27507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: No bootstrap required, opened a new log
I20260807 06:14:46.562438 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:46.545247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.9.133.190:46395)
0807 06:14:46.545864 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:46.545925 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:46.545992 (+ 67us) server_negotiation.cc:407] Connection header received
0807 06:14:46.547459 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:46.547489 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:46.547993 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:46.548446 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:46.549412 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:46.551148 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:46.552237 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:46.552991 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:46.553164 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:46.556505 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:46.556512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:46.556531 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:46.556567 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:46.558806 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:46.559362 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:46.559369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:46.559376 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:46.559529 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:46.559860 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:46.559868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:46.559872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:46.560369 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:46.560542 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:46.560947 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 06:14:46.561252 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":196,"threads_started":1}
I20260807 06:14:46.566885 27507 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:46.567216 27507 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:46.567271 27507 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2cf50b69326447983d59947c5f7f409, State: Initialized, Role: FOLLOWER
I20260807 06:14:46.567708 27507 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:46.567827 27507 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:14:46.567874 27507 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:14:46.567961 27507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:46.569052 27507 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:46.569404 27507 leader_election.cc:302] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2cf50b69326447983d59947c5f7f409; no voters:
I20260807 06:14:46.569777 27507 leader_election.cc:288] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:14:46.569979 27512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:14:46.570322 27512 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 LEADER]: Becoming Leader. State: Replica: c2cf50b69326447983d59947c5f7f409, State: Running, Role: LEADER
I20260807 06:14:46.570779 27512 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:46.571506 27507 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:14:46.573148 27513 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2cf50b69326447983d59947c5f7f409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } } }
I20260807 06:14:46.573333 27513 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:46.573190 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2cf50b69326447983d59947c5f7f409. Latest consensus state: current_term: 1 leader_uuid: "c2cf50b69326447983d59947c5f7f409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } } }
I20260807 06:14:46.573743 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:46.573952 27527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:14:46.574440 27507 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:14:46.583252 27527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:14:46.593293 27527 catalog_manager.cc:1347] Generated new cluster ID: aa8803cbb8bc47659e6fa3d63d1e10fc
I20260807 06:14:46.593384 27527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:14:46.617328 27527 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:14:46.618902 27527 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:14:46.628298 27527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Generated new TSK 0
I20260807 06:14:46.629464 27527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:14:47.535923 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46395
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:47.724510 27574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:47.724838 27574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:47.724888 27574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:47.729526 27574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:47.729631 27574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:47.729745 27574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:14:47.736294 27574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.133.148:36951/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:47.737797 27574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:47.739288 27574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:47.753082 27582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:47.753082 27586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:47.753477 27583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:47.753983 27574 server_base.cc:1034] running on GCE node
I20260807 06:14:47.754555 27574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:47.755497 27574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:47.756690 27574 hybrid_clock.cc:648] HybridClock initialized: now 1786083287756655 us; error 53 us; skew 500 ppm
Aug 07 06:14:47 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083287, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:47.760826 27574 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:14:47.762079 27574 webserver.cc:460] Webserver started at http://127.9.133.129:45209/ using document root <none> and password file <none>
I20260807 06:14:47.762804 27574 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:47.762907 27574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:47.763252 27574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:14:47 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083287, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:47.782646 27574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "fafef438ef8c42329af3e3951e310647"
format_stamp: "Formatted at 2026-08-07 06:14:47 on dist-test-slave-h3ft"
server_key: "b761a4ab843eb9a43617b1bd0f2dbfe0"
server_key_iv: "1d0fa0aab8958216bffc9c4a223645b4"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:47.783505 27574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "fafef438ef8c42329af3e3951e310647"
format_stamp: "Formatted at 2026-08-07 06:14:47 on dist-test-slave-h3ft"
server_key: "b761a4ab843eb9a43617b1bd0f2dbfe0"
server_key_iv: "1d0fa0aab8958216bffc9c4a223645b4"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:47.788828 27574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 06:14:47.833256 27595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:47.835652 27574 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 06:14:47.835839 27574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "fafef438ef8c42329af3e3951e310647"
format_stamp: "Formatted at 2026-08-07 06:14:47 on dist-test-slave-h3ft"
server_key: "b761a4ab843eb9a43617b1bd0f2dbfe0"
server_key_iv: "1d0fa0aab8958216bffc9c4a223645b4"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:47.836036 27574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:47.859717 27574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:47.864565 27574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:47.864869 27574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:47.866053 27574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:47.867575 27574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:47.867687 27574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 06:14:47.867790 27574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:47.867847 27574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:47.889377 27574 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:41743
I20260807 06:14:47.889745 27765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:41743 every 8 connection(s)
I20260807 06:14:47.890772 27574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:14:47.891613 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 27574
I20260807 06:14:47.891757 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:14:47.892198 9750 external_mini_cluster.cc:1428] Setting key 9d4b8e81ae14938e1c3d9b97250795ca
Aug 07 06:14:47 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083287, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:14:47.920703 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:47.894177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:51101 (local address 127.9.133.190:46395)
0807 06:14:47.896240 (+ 2063us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:47.896246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:47.898929 (+ 2683us) server_negotiation.cc:407] Connection header received
0807 06:14:47.898994 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:47.898998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:47.899081 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:47.899185 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:47.901427 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:47.902370 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:47.904379 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:47.904787 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:47.904884 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:47.909453 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:47.909467 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:47.909470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:47.909508 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:47.913886 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:47.914926 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:47.914931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:47.914934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:47.915087 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:47.915705 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:47.915709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:47.915711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:47.915996 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:47.916072 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:47.920211 (+ 4139us) server_negotiation.cc:299] Negotiation successful
0807 06:14:47.920463 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1950,"thread_start_us":164,"threads_started":1}
I20260807 06:14:47.923250 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:47.894814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46395 (local address 127.9.133.129:51101)
0807 06:14:47.896675 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:47.896754 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:47.898528 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:47.899655 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:47.899670 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:47.900297 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:47.901194 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:47.901219 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:47.902575 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:47.902579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:47.904212 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:47.904227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:47.905174 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:47.905179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:47.905522 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:47.906615 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:47.906641 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:47.909121 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:47.914087 (+ 4966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:47.914097 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:47.914126 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:47.914620 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:47.915233 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:47.915237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:47.915239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:47.915536 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:47.916171 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:47.916181 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:47.919688 (+ 3507us) client_negotiation.cc:769] Sending connection context
0807 06:14:47.921095 (+ 1407us) client_negotiation.cc:240] Negotiation successful
0807 06:14:47.921681 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"thread_start_us":236,"threads_started":1}
I20260807 06:14:47.925196 27766 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46395
I20260807 06:14:47.925536 27766 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:47.926345 27766 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:47.928748 27454 ts_manager.cc:194] Registered new tserver with Master: fafef438ef8c42329af3e3951e310647 (127.9.133.129:41743)
I20260807 06:14:47.930789 27454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:51101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:14:47.946929 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46395
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:14:47 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786083287, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 06:14:48.163648 27772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:48.164309 27772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:48.164515 27772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:48.171773 27772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:48.171948 27772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:48.172120 27772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:14:48.181056 27772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.133.148:36951/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:48.183125 27772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:48.184978 27772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:48.201035 27780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:48.201439 27779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:48.201717 27782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:48.204828 27772 server_base.cc:1034] running on GCE node
I20260807 06:14:48.205463 27772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:48.206423 27772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:48.209712 27772 hybrid_clock.cc:648] HybridClock initialized: now 1786083288209669 us; error 68 us; skew 500 ppm
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:48.214432 27772 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:14:48.215929 27772 webserver.cc:460] Webserver started at http://127.9.133.130:33313/ using document root <none> and password file <none>
I20260807 06:14:48.216959 27772 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:48.217111 27772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:48.217448 27772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:48.247272 27772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "bf13befaf149003a32cc85c7d7178b77"
server_key_iv: "220f1d538664ceb243fb8feb3514e0fe"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:48.249269 27772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "bf13befaf149003a32cc85c7d7178b77"
server_key_iv: "220f1d538664ceb243fb8feb3514e0fe"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:48.261130 27772 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.003s sys 0.003s
I20260807 06:14:48.292800 27792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:48.296428 27772 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.003s
I20260807 06:14:48.296752 27772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "bf13befaf149003a32cc85c7d7178b77"
server_key_iv: "220f1d538664ceb243fb8feb3514e0fe"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:48.297075 27772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:48.324613 27772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:48.329978 27772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:48.330236 27772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:48.331118 27772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:48.332530 27772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:48.332623 27772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:48.332692 27772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:48.332724 27772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:48.351763 27772 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:42085
I20260807 06:14:48.351806 27954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:42085 every 8 connection(s)
I20260807 06:14:48.353732 27772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:14:48.359771 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 27772
I20260807 06:14:48.359942 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:14:48.360316 9750 external_mini_cluster.cc:1428] Setting key 953994d0db632a1018e6afedfd3da15d
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:48.393491 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:48.356768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:44203 (local address 127.9.133.190:46395)
0807 06:14:48.356971 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:48.356977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:48.367869 (+ 10892us) server_negotiation.cc:407] Connection header received
0807 06:14:48.370454 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:48.370459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:48.370530 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:48.370618 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:48.373961 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.375395 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.378272 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.378825 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.378975 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:48.384797 (+ 5822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:48.384803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:48.384825 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:48.384874 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:48.387895 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:48.391173 (+ 3278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:48.391177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:48.391179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:48.391301 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:48.391781 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:48.391785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:48.391787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:48.392071 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:48.392155 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:48.393056 (+ 901us) server_negotiation.cc:299] Negotiation successful
0807 06:14:48.393232 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:14:48.395910 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:48.357390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46395 (local address 127.9.133.130:44203)
0807 06:14:48.367586 (+ 10196us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:48.367698 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:48.369994 (+ 2296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:48.371071 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:48.371086 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:48.371898 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:48.373609 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:48.373662 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.375684 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.375690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:48.378048 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:48.378062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.379422 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.379427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:48.379832 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:48.380961 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:48.381014 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:48.384424 (+ 3410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:48.388163 (+ 3739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:48.388172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:48.388200 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:48.388784 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:48.391436 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:48.391439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:48.391442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:48.391662 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:48.392242 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:48.392251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:48.392722 (+ 471us) client_negotiation.cc:769] Sending connection context
0807 06:14:48.394010 (+ 1288us) client_negotiation.cc:240] Negotiation successful
0807 06:14:48.394468 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9744,"thread_start_us":268,"threads_started":1}
I20260807 06:14:48.397773 27957 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46395
I20260807 06:14:48.398102 27957 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:48.398916 27957 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:48.400772 27454 ts_manager.cc:194] Registered new tserver with Master: 84920b5ab1a44c2aa8fb0dc82d2b486c (127.9.133.130:42085)
I20260807 06:14:48.401676 27454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:44203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:14:48.426689 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46395
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083287, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
W20260807 06:14:48.658418 27964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:48.659015 27964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:48.659173 27964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:48.665349 27964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:48.665540 27964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:48.665702 27964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:14:48.674291 27964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.133.148:36951/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:48.676146 27964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:48.677953 27964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:48.694840 27971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:48.694957 27970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:48.696484 27964 server_base.cc:1034] running on GCE node
W20260807 06:14:48.698153 27973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:48.699123 27964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:48.700169 27964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:48.703459 27964 hybrid_clock.cc:648] HybridClock initialized: now 1786083288703406 us; error 157 us; skew 500 ppm
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:48.707762 27964 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:14:48.708915 27964 webserver.cc:460] Webserver started at http://127.9.133.131:43011/ using document root <none> and password file <none>
I20260807 06:14:48.709733 27964 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:48.709833 27964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:48.710167 27964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:48.738488 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "a0952d166f4147ab9f981e66a7e0f5f8"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "813e6389a24c12e3fd18400a49495bde"
server_key_iv: "eee912bad60a23466afdef980039b284"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:48.739415 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "a0952d166f4147ab9f981e66a7e0f5f8"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "813e6389a24c12e3fd18400a49495bde"
server_key_iv: "eee912bad60a23466afdef980039b284"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:48.745144 27964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 06:14:48.773399 27982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:48.775516 27964 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:14:48.775681 27964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "a0952d166f4147ab9f981e66a7e0f5f8"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "813e6389a24c12e3fd18400a49495bde"
server_key_iv: "eee912bad60a23466afdef980039b284"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:48.775866 27964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:48.810923 27964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:48.814944 27964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:48.815281 27964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:48.816226 27964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:48.817550 27964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:48.817623 27964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:48.817770 27964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:48.817817 27964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:48.835632 27964 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:41719
I20260807 06:14:48.836795 27964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:14:48.841974 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 27964
I20260807 06:14:48.842142 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:14:48.842631 9750 external_mini_cluster.cc:1428] Setting key ab1449a3886638c9d7326a20636371f4
I20260807 06:14:48.847775 28149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:41719 every 8 connection(s)
Aug 07 06:14:48 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:48.870568 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:48.850904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:40529 (local address 127.9.133.190:46395)
0807 06:14:48.851109 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:48.851116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:48.852860 (+ 1744us) server_negotiation.cc:407] Connection header received
0807 06:14:48.854834 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:48.854839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:48.854916 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:48.855063 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:48.857233 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.858138 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.860194 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.860564 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.860681 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:48.864775 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:48.864780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:48.864783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:48.864823 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:48.867076 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:48.867935 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:48.867940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:48.867942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:48.868061 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:48.868846 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:48.868850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:48.868853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:48.869156 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:48.869256 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:48.870169 (+ 913us) server_negotiation.cc:299] Negotiation successful
0807 06:14:48.870340 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:14:48.872869 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:48.851693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46395 (local address 127.9.133.131:40529)
0807 06:14:48.852655 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:48.852733 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:48.854498 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:48.855345 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:48.855357 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:48.855980 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:48.857021 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:48.857047 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.858268 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.858271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:48.860037 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:48.860050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:48.860773 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:48.860777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:48.861119 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:48.862028 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:48.862053 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:48.864523 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:48.867244 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:48.867253 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:48.867280 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:48.867748 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:48.868177 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:48.868181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:48.868184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:48.868467 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:48.869341 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:48.869362 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:48.869808 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 06:14:48.870843 (+ 1035us) client_negotiation.cc:240] Negotiation successful
0807 06:14:48.871342 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":275,"threads_started":1}
I20260807 06:14:48.874468 28150 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46395
I20260807 06:14:48.874800 28150 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:48.875656 28150 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:48.877140 27454 ts_manager.cc:194] Registered new tserver with Master: a0952d166f4147ab9f981e66a7e0f5f8 (127.9.133.131:41719)
I20260807 06:14:48.878033 27454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:40529
I20260807 06:14:48.882587 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:14:48.883610 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 27574
I20260807 06:14:48.929289 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 27772
I20260807 06:14:48.950662 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 27964
I20260807 06:14:48.962977 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 27400
I20260807 06:14:48.982286 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46395
--webserver_interface=127.9.133.190
--webserver_port=43503
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:46395
--ranger_config_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:49.120163 28154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:49.120569 28154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:49.120657 28154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:49.125063 28154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:49.125154 28154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:49.125192 28154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:49.125245 28154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:49.125283 28154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:49.131629 28154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:46395
--ranger_config_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=43503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:49.133095 28154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:49.134516 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:14:49.144809 28154 server_base.cc:1034] running on GCE node
W20260807 06:14:49.144873 28164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:49.144832 28161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:49.144799 28162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:49.145887 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:49.147231 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:49.148430 28154 hybrid_clock.cc:648] HybridClock initialized: now 1786083289148409 us; error 35 us; skew 500 ppm
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:49.151753 28154 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:14:49.152873 28154 webserver.cc:460] Webserver started at http://127.9.133.190:43503/ using document root <none> and password file <none>
I20260807 06:14:49.153546 28154 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:49.153656 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:49.238183 28154 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:14:49.241196 28174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:49.242915 28154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:14:49.243144 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "c2cf50b69326447983d59947c5f7f409"
format_stamp: "Formatted at 2026-08-07 06:14:46 on dist-test-slave-h3ft"
server_key: "b9071416305528a193822e9f17583832"
server_key_iv: "670e5af1f4814bfec78c407135bb52e7"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:49.244164 28154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:49.261931 28154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:49.265507 28154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:49.265720 28154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:49.275187 28154 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:46395
I20260807 06:14:49.275195 28257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:46395 every 8 connection(s)
I20260807 06:14:49.276500 28154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:14:49.281952 28259 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:14:49.283553 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 28154
I20260807 06:14:49.284052 28259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Bootstrap starting.
I20260807 06:14:49.284227 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:41743
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:46395
--webserver_port=45209
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:14:49.303653 28259 log.cc:824] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:49.312623 28259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:14:49.313135 28259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Bootstrap complete.
I20260807 06:14:49.318235 28259 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:49.318729 28259 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2cf50b69326447983d59947c5f7f409, State: Initialized, Role: FOLLOWER
I20260807 06:14:49.319499 28259 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:49.319655 28259 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:14:49.319775 28259 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:14:49.319903 28259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:14:49.322419 28259 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:49.322857 28259 leader_election.cc:302] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2cf50b69326447983d59947c5f7f409; no voters:
I20260807 06:14:49.323321 28259 leader_election.cc:288] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:14:49.323441 28266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:14:49.323695 28266 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [term 2 LEADER]: Becoming Leader. State: Replica: c2cf50b69326447983d59947c5f7f409, State: Running, Role: LEADER
I20260807 06:14:49.324061 28266 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } }
I20260807 06:14:49.324973 28259 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:14:49.325326 28266 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2cf50b69326447983d59947c5f7f409. Latest consensus state: current_term: 2 leader_uuid: "c2cf50b69326447983d59947c5f7f409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } } }
I20260807 06:14:49.325479 28266 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:49.325897 28272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:14:49.326009 28268 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c2cf50b69326447983d59947c5f7f409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2cf50b69326447983d59947c5f7f409" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 46395 } } }
I20260807 06:14:49.326215 28268 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:49.330108 28272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:14:49.330919 28272 catalog_manager.cc:1259] Loaded cluster ID: aa8803cbb8bc47659e6fa3d63d1e10fc
I20260807 06:14:49.331120 28259 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 06:14:49.330996 28272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:14:49.342370 28272 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:14:49.343864 28272 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Loaded TSK: 0
I20260807 06:14:49.345607 28272 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:14:49.437186 28263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:49.437537 28263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:49.437618 28263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:49.441953 28263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:49.442044 28263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:49.442152 28263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:14:49.448567 28263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:41743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=45209
--tserver_master_addrs=127.9.133.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:49.450381 28263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:49.451829 28263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:49.466645 28316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:49.467782 28312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:49.468544 28313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:49.468986 28263 server_base.cc:1034] running on GCE node
I20260807 06:14:49.469529 28263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:49.470556 28263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:49.471761 28263 hybrid_clock.cc:648] HybridClock initialized: now 1786083289471701 us; error 57 us; skew 500 ppm
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:49.475426 28263 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:14:49.476645 28263 webserver.cc:460] Webserver started at http://127.9.133.129:45209/ using document root <none> and password file <none>
I20260807 06:14:49.477334 28263 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:49.477447 28263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:49.508178 28263 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 06:14:49.511773 28327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:49.513432 28263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 06:14:49.513610 28263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "fafef438ef8c42329af3e3951e310647"
format_stamp: "Formatted at 2026-08-07 06:14:47 on dist-test-slave-h3ft"
server_key: "b761a4ab843eb9a43617b1bd0f2dbfe0"
server_key_iv: "1d0fa0aab8958216bffc9c4a223645b4"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:49.514328 28263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:49.544138 28263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:49.548924 28263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:49.549196 28263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:49.550175 28263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:49.551527 28263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:49.551594 28263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:49.551652 28263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:49.551680 28263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:49.565044 28263 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:41743
I20260807 06:14:49.566709 28263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:14:49.569142 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 28263
I20260807 06:14:49.569744 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:42085
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:46395
--webserver_port=33313
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:14:49.585754 28498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:41743 every 8 connection(s)
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:49.613982 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:49.583063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46395 (local address 127.9.133.129:49227)
0807 06:14:49.586885 (+ 3822us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:49.586976 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:49.589518 (+ 2542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:49.593669 (+ 4151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:49.593683 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:49.594149 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:49.594955 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:49.594981 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.597274 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.597279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:49.598983 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:49.598996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.600474 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.600479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:49.600690 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:49.603247 (+ 2557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:49.603277 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:49.605584 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:49.609047 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:49.609055 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:49.609073 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:49.609530 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:49.610040 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:49.610044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:49.610046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:49.610267 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:49.611341 (+ 1074us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:49.611349 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:49.611739 (+ 390us) client_negotiation.cc:769] Sending connection context
0807 06:14:49.611996 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 06:14:49.612442 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2989,"thread_start_us":235,"threads_started":1}
I20260807 06:14:49.617904 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:49.585082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:49227 (local address 127.9.133.190:46395)
0807 06:14:49.591425 (+ 6343us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:49.591457 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:49.591522 (+ 65us) server_negotiation.cc:407] Connection header received
0807 06:14:49.591931 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:49.591950 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:49.592517 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:49.593005 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:49.595224 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.597081 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.599165 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.599992 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.600185 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:49.605899 (+ 5714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:49.605923 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:49.605947 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:49.605991 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:49.608862 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:49.609706 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:49.609714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:49.609721 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:49.609896 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:49.610428 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:49.610432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:49.610434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:49.611118 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:49.611275 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:49.616349 (+ 5074us) server_negotiation.cc:299] Negotiation successful
0807 06:14:49.616760 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6109,"thread_start_us":157,"threads_started":1}
I20260807 06:14:49.622382 28499 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46395
I20260807 06:14:49.622713 28499 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:49.623489 28499 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
W20260807 06:14:49.626490 28499 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:46395 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:14:49.626648 28499 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:49.626879 28499 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:49.628129 28499 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:49.628321 28499 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
W20260807 06:14:49.629333 28499 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:14:49.765276 28502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:49.765760 28502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:49.765849 28502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:49.771400 28502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:49.771474 28502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:49.771571 28502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:14:49.778031 28502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:42085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=33313
--tserver_master_addrs=127.9.133.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:49.779810 28502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:49.781168 28502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:49.793232 28520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:49.793272 28518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:49.793236 28517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:49.793700 28502 server_base.cc:1034] running on GCE node
I20260807 06:14:49.794188 28502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:49.795107 28502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:49.796304 28502 hybrid_clock.cc:648] HybridClock initialized: now 1786083289796270 us; error 56 us; skew 500 ppm
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:49.800690 28502 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:14:49.801991 28502 webserver.cc:460] Webserver started at http://127.9.133.130:33313/ using document root <none> and password file <none>
I20260807 06:14:49.802742 28502 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:49.802870 28502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:49.841986 28502 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.001s
I20260807 06:14:49.845948 28531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:49.848129 28502 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 06:14:49.848300 28502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "bf13befaf149003a32cc85c7d7178b77"
server_key_iv: "220f1d538664ceb243fb8feb3514e0fe"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:49.849180 28502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:49.869544 28502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:49.873955 28502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:49.874230 28502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:49.875115 28502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:49.876474 28502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:49.876547 28502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:49.876631 28502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:49.876664 28502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 06:14:49.889278 28502 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:42085
I20260807 06:14:49.889654 28697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:42085 every 8 connection(s)
I20260807 06:14:49.890864 28502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:14:49.898272 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 28502
I20260807 06:14:49.898980 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.133.148:36951/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:41719
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:46395
--webserver_port=43011
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:14:49 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083289, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:49.911599 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:49.893663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:40823 (local address 127.9.133.190:46395)
0807 06:14:49.893853 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:49.893858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:49.895833 (+ 1975us) server_negotiation.cc:407] Connection header received
0807 06:14:49.897317 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:49.897322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:49.897410 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:49.897478 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:49.899346 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.900233 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.902134 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.902510 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.902605 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:49.906358 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:49.906363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:49.906366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:49.906403 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:49.908477 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:49.909237 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:49.909241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:49.909243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:49.909346 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:49.909844 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:49.909848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:49.909850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:49.910156 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:49.910254 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:49.911192 (+ 938us) server_negotiation.cc:299] Negotiation successful
0807 06:14:49.911375 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 06:14:49.914126 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:49.894197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46395 (local address 127.9.133.130:40823)
0807 06:14:49.895621 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:49.895686 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:49.897005 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:49.897912 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:49.897920 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:49.898260 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:49.899133 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:49.899164 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.900416 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.900421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:49.901970 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:49.901993 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:49.902880 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:49.902886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:49.903145 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:49.903972 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:49.904009 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:49.906102 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:49.908632 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:49.908642 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:49.908667 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:49.909103 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:49.909469 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:49.909474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:49.909475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:49.909713 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:49.910331 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:49.910338 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:49.910779 (+ 441us) client_negotiation.cc:769] Sending connection context
0807 06:14:49.912255 (+ 1476us) client_negotiation.cc:240] Negotiation successful
0807 06:14:49.912735 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1163,"thread_start_us":145,"threads_started":1}
I20260807 06:14:49.916242 28698 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46395
I20260807 06:14:49.916582 28698 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:49.917377 28698 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
W20260807 06:14:49.919709 28698 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:46395 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 06:14:49.919857 28698 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:49.920111 28698 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:49.921241 28698 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:49.921449 28698 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
W20260807 06:14:49.922415 28698 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:14:50.102946 28702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:50.103583 28702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:50.103684 28702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:50.110879 28702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:50.110987 28702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:50.111123 28702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:14:50.120645 28702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.133.148:36951/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:41719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=43011
--tserver_master_addrs=127.9.133.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:50.122611 28702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:50.124387 28702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:50.139875 28712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:50.139816 28716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:50.140435 28713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:50.141410 28702 server_base.cc:1034] running on GCE node
I20260807 06:14:50.141983 28702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:50.143069 28702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:50.144277 28702 hybrid_clock.cc:648] HybridClock initialized: now 1786083290144240 us; error 50 us; skew 500 ppm
Aug 07 06:14:50 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083290, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:50.148228 28702 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:14:50.149452 28702 webserver.cc:460] Webserver started at http://127.9.133.131:43011/ using document root <none> and password file <none>
I20260807 06:14:50.150213 28702 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:50.150300 28702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 06:14:50 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083290, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
I20260807 06:14:50.175065 28702 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 06:14:50.179318 28725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:50.181465 28702 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 06:14:50.181701 28702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "a0952d166f4147ab9f981e66a7e0f5f8"
format_stamp: "Formatted at 2026-08-07 06:14:48 on dist-test-slave-h3ft"
server_key: "813e6389a24c12e3fd18400a49495bde"
server_key_iv: "eee912bad60a23466afdef980039b284"
server_key_version: "kuduclusterkey@0"
I20260807 06:14:50.182662 28702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:50.213083 28702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:50.217676 28702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:50.217962 28702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:50.219022 28702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:50.220414 28702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:50.220492 28702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:50.220566 28702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:50.220603 28702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:50.236868 28702 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:41719
I20260807 06:14:50.237108 28896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:41719 every 8 connection(s)
I20260807 06:14:50.238387 28702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:14:50.242940 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 28702
Aug 07 06:14:50 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083290, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:50.256621 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.243887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.9.133.190:46395)
0807 06:14:50.244735 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:50.244745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:50.244756 (+ 11us) server_negotiation.cc:407] Connection header received
0807 06:14:50.244781 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:50.244784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:50.244829 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:50.244927 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:50.246882 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.247786 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.248898 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.249319 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.249444 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.251673 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:50.251679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:50.251684 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:50.251715 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:50.254129 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:50.254738 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:50.254745 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:50.254750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:50.254868 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:50.255276 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:50.255282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:50.255284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:50.255706 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:50.255980 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:50.256180 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 06:14:50.256331 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1834,"server-negotiator.queue_time_us":749,"spinlock_wait_cycles":11520,"thread_start_us":141,"threads_started":1}
I20260807 06:14:50.258414 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.243093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:38841 (local address 127.9.133.190:46395)
0807 06:14:50.243294 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:50.243299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:50.243321 (+ 22us) server_negotiation.cc:407] Connection header received
0807 06:14:50.244762 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:50.244767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:50.244828 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:50.244897 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:50.246782 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.247786 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.250556 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.250788 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.250902 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.253986 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:50.253992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:50.253998 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:50.254028 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:50.255778 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:50.256499 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:50.256503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:50.256504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:50.256584 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:50.257042 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:50.257046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:50.257047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:50.257277 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:50.257362 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:50.258154 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 06:14:50.258274 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":107}
I20260807 06:14:50.259812 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.241567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:46395 (local address 127.9.133.131:38841)
0807 06:14:50.242152 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:50.242214 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:50.244473 (+ 2259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:50.245507 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:50.245522 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:50.245917 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:50.246606 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.246623 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.248956 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.248960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.250416 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.250428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.250967 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.250971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.251212 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.251901 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:50.251929 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:50.253756 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:50.255910 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:50.255917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:50.255939 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:50.256380 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:50.256703 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:50.256706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:50.256708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:50.256926 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:50.257445 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:50.257452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:50.257866 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 06:14:50.258114 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 06:14:50.258545 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":174,"threads_started":1}
I20260807 06:14:50.261268 28897 heartbeater.cc:347] Connected to a master server at 127.9.133.190:46395
I20260807 06:14:50.261629 28897 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:50.262411 28897 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:50.264297 28200 ts_manager.cc:194] Registered new tserver with Master: a0952d166f4147ab9f981e66a7e0f5f8 (127.9.133.131:41719)
I20260807 06:14:50.265679 28200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:38841
I20260807 06:14:50.629619 28499 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:50.630123 28499 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:50.631278 28200 ts_manager.cc:194] Registered new tserver with Master: fafef438ef8c42329af3e3951e310647 (127.9.133.129:41743)
I20260807 06:14:50.632062 28200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:49227
Aug 07 06:14:50 dist-test-slave-h3ft krb5kdc[26754](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786083290, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:50.922811 28698 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:50.923394 28698 heartbeater.cc:511] Master 127.9.133.190:46395 requested a full tablet report, sending...
I20260807 06:14:50.924418 28200 ts_manager.cc:194] Registered new tserver with Master: 84920b5ab1a44c2aa8fb0dc82d2b486c (127.9.133.130:42085)
I20260807 06:14:50.925321 28200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:40823
I20260807 06:14:50.937546 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:14:50.950723 28923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.940029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.9.133.190:46395)
0807 06:14:50.940420 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:50.940425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:50.940441 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:14:50.940626 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:50.940630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:50.940697 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:50.940812 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:50.941593 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.942274 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.943628 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.944032 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.944136 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.946172 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:50.946186 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:50.946189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:50.946223 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:50.948350 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:50.949033 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:50.949038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:50.949040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:50.949142 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:50.949483 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:50.949486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:50.949488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:50.949773 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:50.949936 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:50.950308 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 06:14:50.950479 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":87,"threads_started":1}
I20260807 06:14:50.955849 28200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:14:50.960908 28200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:14:50.986249 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.977175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:41743 (local address 127.0.0.1:41736)
0807 06:14:50.977723 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:50.977791 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:50.977967 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:50.979942 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:50.979950 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:50.979979 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:14:50.980169 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.980177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.981792 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.981796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.984362 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.984374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.985623 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.985627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.985767 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.985820 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 06:14:50.985917 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 06:14:50.986064 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":164,"threads_started":1}
I20260807 06:14:50.988483 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.977389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.9.133.129:41743)
0807 06:14:50.979040 (+ 1651us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:50.979049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:50.979082 (+ 33us) server_negotiation.cc:407] Connection header received
0807 06:14:50.979188 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:50.979195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:50.979483 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:50.979695 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:14:50.980338 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.981636 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.984546 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.985468 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.987796 (+ 2328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.988046 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:50.988113 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:14:50.988294 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1479,"thread_start_us":191,"threads_started":1}
I20260807 06:14:50.993125 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.978924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:41719 (local address 127.0.0.1:52946)
0807 06:14:50.980782 (+ 1858us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:50.980797 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:50.980881 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:50.983040 (+ 2159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:50.983043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:50.983062 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:14:50.983181 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.983189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.986598 (+ 3409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.986602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.988291 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.988304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.992621 (+ 4317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.992626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.992741 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.992773 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 06:14:50.992833 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 06:14:50.992941 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1769,"thread_start_us":168,"threads_started":1}
I20260807 06:14:50.993973 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.978682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:42085 (local address 127.0.0.1:53946)
0807 06:14:50.980453 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:50.980469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:50.980564 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:50.981885 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:50.981889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:50.981908 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:14:50.982026 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.982034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.988474 (+ 6440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.988477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.989744 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:50.989758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.993381 (+ 3623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.993385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:50.993625 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.993656 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:14:50.993728 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 06:14:50.993808 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1686,"thread_start_us":108,"threads_started":1}
I20260807 06:14:50.994910 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.979494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.9.133.131:41719)
0807 06:14:50.982324 (+ 2830us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:50.982331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:50.982356 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:14:50.982464 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:50.982471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:50.982671 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:50.982829 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:14:50.985067 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.986428 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.991432 (+ 5004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.992436 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.994295 (+ 1859us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.994517 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:50.994599 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 06:14:50.994734 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2708,"thread_start_us":140,"threads_started":1}
I20260807 06:14:50.995738 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:50.979226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.9.133.130:42085)
0807 06:14:50.981157 (+ 1931us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:50.981165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:50.981206 (+ 41us) server_negotiation.cc:407] Connection header received
0807 06:14:50.981298 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:50.981304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:50.981533 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:50.981700 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:14:50.983334 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.984890 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.990117 (+ 5227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:50.991184 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:50.995142 (+ 3958us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:50.995337 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:50.995403 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 06:14:50.995549 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1804,"thread_start_us":116,"threads_started":1}
I20260807 06:14:50.993155 28399 tablet_service.cc:1467] Processing CreateTablet for tablet fff4a4e841e44c8fa3dc0be769e602cf (DEFAULT_TABLE table=test-table [id=d3979cb751eb475597656dcfbf246797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:14:50.997231 28399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fff4a4e841e44c8fa3dc0be769e602cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:50.998823 28604 tablet_service.cc:1467] Processing CreateTablet for tablet fff4a4e841e44c8fa3dc0be769e602cf (DEFAULT_TABLE table=test-table [id=d3979cb751eb475597656dcfbf246797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:14:50.998824 28798 tablet_service.cc:1467] Processing CreateTablet for tablet fff4a4e841e44c8fa3dc0be769e602cf (DEFAULT_TABLE table=test-table [id=d3979cb751eb475597656dcfbf246797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:14:51.000170 28604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fff4a4e841e44c8fa3dc0be769e602cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:51.000172 28798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fff4a4e841e44c8fa3dc0be769e602cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:51.012166 28943 tablet_bootstrap.cc:492] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Bootstrap starting.
I20260807 06:14:51.015540 28945 tablet_bootstrap.cc:492] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: Bootstrap starting.
I20260807 06:14:51.015681 28943 tablet_bootstrap.cc:654] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:51.016956 28943 log.cc:824] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:51.018026 28944 tablet_bootstrap.cc:492] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Bootstrap starting.
I20260807 06:14:51.020710 28945 tablet_bootstrap.cc:654] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:51.020766 28944 tablet_bootstrap.cc:654] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:51.021908 28945 log.cc:824] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:51.021919 28944 log.cc:824] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:51.025879 28943 tablet_bootstrap.cc:492] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: No bootstrap required, opened a new log
I20260807 06:14:51.026320 28943 ts_tablet_manager.cc:1397] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260807 06:14:51.027616 28944 tablet_bootstrap.cc:492] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: No bootstrap required, opened a new log
I20260807 06:14:51.027868 28944 ts_tablet_manager.cc:1397] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 06:14:51.031396 28945 tablet_bootstrap.cc:492] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: No bootstrap required, opened a new log
I20260807 06:14:51.031706 28945 ts_tablet_manager.cc:1397] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.002s
I20260807 06:14:51.033243 28944 raft_consensus.cc:348] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.033651 28944 raft_consensus.cc:374] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:51.033718 28944 raft_consensus.cc:729] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84920b5ab1a44c2aa8fb0dc82d2b486c, State: Initialized, Role: FOLLOWER
I20260807 06:14:51.034224 28943 raft_consensus.cc:348] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.034428 28944 consensus_queue.cc:260] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.034765 28943 raft_consensus.cc:374] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:51.034888 28943 raft_consensus.cc:729] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0952d166f4147ab9f981e66a7e0f5f8, State: Initialized, Role: FOLLOWER
I20260807 06:14:51.035486 28943 consensus_queue.cc:260] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.035897 28944 ts_tablet_manager.cc:1428] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 06:14:51.036408 28698 heartbeater.cc:503] Master 127.9.133.190:46395 was elected leader, sending a full tablet report...
I20260807 06:14:51.036330 28945 raft_consensus.cc:348] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.036644 28945 raft_consensus.cc:374] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:51.036712 28945 raft_consensus.cc:729] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fafef438ef8c42329af3e3951e310647, State: Initialized, Role: FOLLOWER
I20260807 06:14:51.037230 28943 ts_tablet_manager.cc:1428] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260807 06:14:51.037699 28897 heartbeater.cc:503] Master 127.9.133.190:46395 was elected leader, sending a full tablet report...
I20260807 06:14:51.038127 28945 consensus_queue.cc:260] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.039557 28945 ts_tablet_manager.cc:1428] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260807 06:14:51.040115 28499 heartbeater.cc:503] Master 127.9.133.190:46395 was elected leader, sending a full tablet report...
W20260807 06:14:51.081534 28500 tablet.cc:2369] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 06:14:51 dist-test-slave-h3ft krb5kdc[26754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083290, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for HTTP/127.9.133.148@KRBTEST.COM
W20260807 06:14:51.143671 28699 tablet.cc:2369] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:14:51.231976 28959 raft_consensus.cc:482] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:14:51.232347 28959 raft_consensus.cc:504] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.234062 28959 leader_election.cc:288] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84920b5ab1a44c2aa8fb0dc82d2b486c (127.9.133.130:42085), a0952d166f4147ab9f981e66a7e0f5f8 (127.9.133.131:41719)
I20260807 06:14:51.240578 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.234924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:42007 (local address 127.9.133.131:41719)
0807 06:14:51.235111 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:51.235117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:51.235485 (+ 368us) server_negotiation.cc:407] Connection header received
0807 06:14:51.235752 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:51.235758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:51.235831 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:51.235946 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:14:51.236364 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.237179 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.238931 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.239637 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.239755 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:51.239826 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:51.240309 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 06:14:51.240425 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 06:14:51.241070 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.234404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:41719 (local address 127.9.133.129:42007)
0807 06:14:51.235395 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:51.235410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:51.235578 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:51.235989 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:51.235994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:51.236019 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:14:51.236196 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:51.236206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.237354 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.237358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:51.238757 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:51.238769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.240024 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.240028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:51.240148 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:51.240177 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:14:51.240780 (+ 603us) client_negotiation.cc:240] Negotiation successful
0807 06:14:51.240869 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":34,"spinlock_wait_cycles":66560,"thread_start_us":141,"threads_started":1}
I20260807 06:14:51.241916 28807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff4a4e841e44c8fa3dc0be769e602cf" candidate_uuid: "fafef438ef8c42329af3e3951e310647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" is_pre_election: true
I20260807 06:14:51.242277 28807 raft_consensus.cc:2454] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafef438ef8c42329af3e3951e310647 in term 0.
I20260807 06:14:51.242942 28328 leader_election.cc:302] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0952d166f4147ab9f981e66a7e0f5f8, fafef438ef8c42329af3e3951e310647; no voters:
I20260807 06:14:51.243328 28959 raft_consensus.cc:2781] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:14:51.243427 28959 raft_consensus.cc:482] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:14:51.243469 28959 raft_consensus.cc:3037] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 0 FOLLOWER]: Advancing to term 1
W20260807 06:14:51.244760 28898 tablet.cc:2369] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:14:51.245177 28959 raft_consensus.cc:504] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.245631 28959 leader_election.cc:288] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [CANDIDATE]: Term 1 election: Requested vote from peers 84920b5ab1a44c2aa8fb0dc82d2b486c (127.9.133.130:42085), a0952d166f4147ab9f981e66a7e0f5f8 (127.9.133.131:41719)
I20260807 06:14:51.246086 28807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff4a4e841e44c8fa3dc0be769e602cf" candidate_uuid: "fafef438ef8c42329af3e3951e310647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0952d166f4147ab9f981e66a7e0f5f8"
I20260807 06:14:51.246217 28807 raft_consensus.cc:3037] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:51.247594 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.234331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:42085 (local address 127.9.133.129:55341)
0807 06:14:51.235336 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:51.235354 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:51.235578 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:51.236731 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:51.236736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:51.236755 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:14:51.236893 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:51.236899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.237936 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.237939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:51.246143 (+ 8204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:51.246155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.247056 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.247060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:51.247173 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:51.247202 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:14:51.247308 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:14:51.247400 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":6996,"thread_start_us":186,"threads_started":1}
I20260807 06:14:51.247591 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.234938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:55341 (local address 127.9.133.130:42085)
0807 06:14:51.235111 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:51.235117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:51.235462 (+ 345us) server_negotiation.cc:407] Connection header received
0807 06:14:51.236465 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:51.236468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:51.236529 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:51.236579 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:14:51.237022 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.237761 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.246382 (+ 8621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.246852 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.247050 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:51.247128 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:51.247353 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 06:14:51.247453 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:14:51.247658 28807 raft_consensus.cc:2454] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafef438ef8c42329af3e3951e310647 in term 1.
I20260807 06:14:51.248051 28328 leader_election.cc:302] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0952d166f4147ab9f981e66a7e0f5f8, fafef438ef8c42329af3e3951e310647; no voters:
I20260807 06:14:51.248333 28959 raft_consensus.cc:2781] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:14:51.248663 28959 raft_consensus.cc:686] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [term 1 LEADER]: Becoming Leader. State: Replica: fafef438ef8c42329af3e3951e310647, State: Running, Role: LEADER
I20260807 06:14:51.249155 28959 consensus_queue.cc:237] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } }
I20260807 06:14:51.249774 28620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff4a4e841e44c8fa3dc0be769e602cf" candidate_uuid: "fafef438ef8c42329af3e3951e310647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" is_pre_election: true
I20260807 06:14:51.250124 28620 raft_consensus.cc:2454] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fafef438ef8c42329af3e3951e310647 in term 0.
I20260807 06:14:51.249790 28622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff4a4e841e44c8fa3dc0be769e602cf" candidate_uuid: "fafef438ef8c42329af3e3951e310647" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c"
I20260807 06:14:51.250559 28622 raft_consensus.cc:3037] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:51.252540 28622 raft_consensus.cc:2454] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fafef438ef8c42329af3e3951e310647 in term 1.
I20260807 06:14:51.254016 28195 catalog_manager.cc:5495] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 reported cstate change: term changed from 0 to 1, leader changed from <none> to fafef438ef8c42329af3e3951e310647 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "fafef438ef8c42329af3e3951e310647" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fafef438ef8c42329af3e3951e310647" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 41743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 } health_report { overall_health: UNKNOWN } } }
I20260807 06:14:51.286305 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.281917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.9.133.129:41743)
0807 06:14:51.282119 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:51.282125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:51.282149 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:14:51.282290 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:51.282295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:51.282398 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:51.282578 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:14:51.282856 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.283566 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.284509 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:51.284746 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:51.284863 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:51.285215 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:14:51.285825 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:14:51.285987 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:51.286021 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 06:14:51.286090 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:14:51.296147 28807 raft_consensus.cc:1261] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 1 FOLLOWER]: Refusing update from remote peer fafef438ef8c42329af3e3951e310647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:14:51.296387 28622 raft_consensus.cc:1261] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 1 FOLLOWER]: Refusing update from remote peer fafef438ef8c42329af3e3951e310647: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:14:51.297264 28959 consensus_queue.cc:1035] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84920b5ab1a44c2aa8fb0dc82d2b486c" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 42085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:14:51.297571 28962 consensus_queue.cc:1035] T fff4a4e841e44c8fa3dc0be769e602cf P fafef438ef8c42329af3e3951e310647 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0952d166f4147ab9f981e66a7e0f5f8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 41719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:14:51.310233 28967 mvcc.cc:205] Tried to move back new op lower bound from 7315797161138368512 to 7315797160960495616. Current Snapshot: MvccSnapshot[applied={T|T < 7315797161138368512}]
I20260807 06:14:51.318871 28968 mvcc.cc:205] Tried to move back new op lower bound from 7315797161138368512 to 7315797160960495616. Current Snapshot: MvccSnapshot[applied={T|T < 7315797161138368512}]
I20260807 06:14:51.323659 28200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:14:51.323900 28200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:14:51.327695 28200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2cf50b69326447983d59947c5f7f409: Sending DeleteTablet for 3 replicas of tablet fff4a4e841e44c8fa3dc0be769e602cf
I20260807 06:14:51.328788 28798 tablet_service.cc:1514] Processing DeleteTablet for tablet fff4a4e841e44c8fa3dc0be769e602cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:14:51 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:52946
I20260807 06:14:51.329221 28399 tablet_service.cc:1514] Processing DeleteTablet for tablet fff4a4e841e44c8fa3dc0be769e602cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:14:51 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:41736
I20260807 06:14:51.329833 28604 tablet_service.cc:1514] Processing DeleteTablet for tablet fff4a4e841e44c8fa3dc0be769e602cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:14:51 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:53946
I20260807 06:14:51.331677 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 28263
I20260807 06:14:51.340232 28985 tablet_replica.cc:330] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: stopping tablet replica
I20260807 06:14:51.340869 28985 raft_consensus.cc:2229] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:14:51.341477 28985 raft_consensus.cc:2258] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:14:51.344019 28985 ts_tablet_manager.cc:1905] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:14:51.345459 28982 tablet_replica.cc:330] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: stopping tablet replica
I20260807 06:14:51.345964 28982 raft_consensus.cc:2229] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:14:51.347451 28982 raft_consensus.cc:2258] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:14:51.350278 28985 ts_tablet_manager.cc:1918] T fff4a4e841e44c8fa3dc0be769e602cf P 84920b5ab1a44c2aa8fb0dc82d2b486c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 06:14:51.350564 28178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:41743 (error 108)
I20260807 06:14:51.352309 28982 ts_tablet_manager.cc:1905] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:14:51.352866 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.352405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:41743 (local address 127.0.0.1:41754)
0807 06:14:51.352662 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:51.352746 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:41743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260807 06:14:51.353093 28178 catalog_manager.cc:4584] TS fafef438ef8c42329af3e3951e310647 (127.9.133.129:41743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fff4a4e841e44c8fa3dc0be769e602cf: Network error: Client connection negotiation failed: client connection to 127.9.133.129:41743: connect: Connection refused (error 111)
I20260807 06:14:51.353556 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 28502
I20260807 06:14:51.356917 28982 ts_tablet_manager.cc:1918] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:14:51.357112 28982 log.cc:1196] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/fff4a4e841e44c8fa3dc0be769e602cf
I20260807 06:14:51.357614 28982 ts_tablet_manager.cc:1939] T fff4a4e841e44c8fa3dc0be769e602cf P a0952d166f4147ab9f981e66a7e0f5f8: Deleting consensus metadata
I20260807 06:14:51.359539 28176 catalog_manager.cc:4841] TS a0952d166f4147ab9f981e66a7e0f5f8 (127.9.133.131:41719): tablet fff4a4e841e44c8fa3dc0be769e602cf (table test-table [id=d3979cb751eb475597656dcfbf246797]) successfully deleted
I20260807 06:14:51.366806 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 28702
I20260807 06:14:51.373864 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:51.373319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:42085 (local address 127.0.0.1:53958)
0807 06:14:51.373631 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:51.373715 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.130:42085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260807 06:14:51.380402 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 28154
I20260807 06:14:51.398353 9750 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 06:14:51.745757 9750 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 06:14:51.746161 9750 mini_ranger.cc:66] Stopping Ranger...
I20260807 06:14:52.102370 9750 mini_ranger.cc:68] Stopped Ranger
2026-08-07 06:14:52.102 UTC [26794] LOG: received smart shutdown request
2026-08-07 06:14:52.108 UTC [26794] LOG: background worker "logical replication launcher" (PID 26805) exited with exit code 1
2026-08-07 06:14:52.108 UTC [26799] LOG: shutting down
2026-08-07 06:14:52.290 UTC [26794] LOG: database system is shut down
2026-08-07T06:14:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (61732 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:14:52 dist-test-slave-h3ft krb5kdc[29002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:14:52 dist-test-slave-h3ft krb5kdc[29002](info): set up 2 sockets
Aug 07 06:14:52 dist-test-slave-h3ft krb5kdc[29002](info): commencing operation
krb5kdc: starting...
W20260807 06:14:53.710567 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:14:53 dist-test-slave-h3ft krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:14:53.871975 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42203
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42203,127.9.133.189:34537,127.9.133.188:35939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:54.011164 29023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:54.011615 29023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:54.011709 29023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:54.016132 29023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:54.016265 29023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:54.016304 29023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:54.016348 29023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:54.016386 29023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:54.022387 29023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42203,127.9.133.189:34537,127.9.133.188:35939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:54.024037 29023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:54.025465 29023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:54.036183 29030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:54.036139 29029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:54.036302 29023 server_base.cc:1034] running on GCE node
W20260807 06:14:54.036206 29032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:54.037297 29023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:54.038704 29023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:54.039907 29023 hybrid_clock.cc:648] HybridClock initialized: now 1786083294039880 us; error 44 us; skew 500 ppm
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:54.043356 29023 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:14:54.044497 29023 webserver.cc:460] Webserver started at http://127.9.133.190:45629/ using document root <none> and password file <none>
I20260807 06:14:54.045199 29023 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:54.045301 29023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:54.045589 29023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:54.048236 29023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "a2eac702e31d44bc92ff98938fdf6c17"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "59a0a6dafff2f8493547df94090188f7"
server_key_iv: "f8d3993d9a6c3646e727cdba1090176f"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.048916 29023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "a2eac702e31d44bc92ff98938fdf6c17"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "59a0a6dafff2f8493547df94090188f7"
server_key_iv: "f8d3993d9a6c3646e727cdba1090176f"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.053504 29023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:14:54.056547 29042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:54.058039 29023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:14:54.058212 29023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "a2eac702e31d44bc92ff98938fdf6c17"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "59a0a6dafff2f8493547df94090188f7"
server_key_iv: "f8d3993d9a6c3646e727cdba1090176f"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.058342 29023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:54.072093 29023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:54.076182 29023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:54.076438 29023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:54.085671 29023 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:42203
I20260807 06:14:54.085703 29120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:42203 every 8 connection(s)
I20260807 06:14:54.086988 29023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:14:54.090808 29121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:54.092646 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29023
I20260807 06:14:54.092823 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:14:54.093221 9750 external_mini_cluster.cc:1428] Setting key 738a8cf0d5d8d2631f6df5be232ba2dd
I20260807 06:14:54.096143 29121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.133.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.189@KRBTEST.COM" created.
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:54.115088 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.099337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42203 (local address 127.0.0.1:42868)
0807 06:14:54.100092 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.100141 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.101225 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.102062 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.102067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.102616 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.102915 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.102931 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.104295 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.104303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.105169 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.105180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.105926 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.105930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.106104 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.106868 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.106889 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.108831 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.111499 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.111513 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.111530 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.111902 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.112290 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.112296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.112300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.112498 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.113231 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.113238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.113432 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.113599 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.113962 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":4,"spinlock_wait_cycles":12032,"thread_start_us":224,"threads_started":1}
I20260807 06:14:54.115096 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.099337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.9.133.190:42203)
0807 06:14:54.099979 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.100009 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.100284 (+ 275us) server_negotiation.cc:407] Connection header received
0807 06:14:54.101608 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.101613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.101834 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.102065 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.103156 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.104149 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.105343 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.105749 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.106010 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.109023 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.109034 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.109054 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.109090 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.111370 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.112043 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.112050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.112051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.112184 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.112613 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.112616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.112617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.113112 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.113236 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.113755 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.114089 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":396,"thread_start_us":182,"threads_started":1}
I20260807 06:14:54.118304 29121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:14:54.119331 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.118983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53232)
0807 06:14:54.119167 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.119255 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 06:14:54.119618 29044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
I20260807 06:14:54.120646 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.120425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53234)
0807 06:14:54.120541 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.120590 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 06:14:54.120934 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34537: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab.
Entry for principal kudu/127.9.133.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab.
I20260807 06:14:54.131637 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:34537
--webserver_interface=127.9.133.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab
--principal=kudu/127.9.133.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42203,127.9.133.189:34537,127.9.133.188:35939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:14:54.170284 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } attempt: 1
I20260807 06:14:54.171551 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.171152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53246)
0807 06:14:54.171362 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.171441 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 06:14:54.172426 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.172180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53262)
0807 06:14:54.172300 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.172358 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 06:14:54.172595 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34537: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
I20260807 06:14:54.240792 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } attempt: 2
I20260807 06:14:54.242000 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.241560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53278)
0807 06:14:54.241771 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.241858 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 06:14:54.243326 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.243053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53290)
0807 06:14:54.243195 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.243258 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 06:14:54.243575 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34537: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
W20260807 06:14:54.271561 29134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:54.271976 29134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:54.272023 29134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:54.276489 29134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:54.276592 29134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:54.276615 29134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:54.276638 29134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:54.276697 29134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:54.282691 29134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42203,127.9.133.189:34537,127.9.133.188:35939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab
--principal=kudu/127.9.133.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:34537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.133.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:54.284298 29134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:54.285662 29134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:54.296005 29141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:54.296084 29144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:54.296005 29142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:54.296665 29134 server_base.cc:1034] running on GCE node
I20260807 06:14:54.297410 29134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:54.298712 29134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:54.299958 29134 hybrid_clock.cc:648] HybridClock initialized: now 1786083294299931 us; error 37 us; skew 500 ppm
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:54.303515 29134 init.cc:377] Logged in from keytab as kudu/127.9.133.189@KRBTEST.COM (short username kudu)
I20260807 06:14:54.304677 29134 webserver.cc:460] Webserver started at http://127.9.133.189:45505/ using document root <none> and password file <none>
I20260807 06:14:54.305413 29134 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:54.305491 29134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:54.305770 29134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:54.308557 29134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data/instance:
uuid: "781b58e8aadf4eaaab3866230b96d4e3"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "4fc15d21b746abcfbb94f4dc54f66781"
server_key_iv: "2aed631b3f03a642509744c3ac7dd58f"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.309351 29134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal/instance:
uuid: "781b58e8aadf4eaaab3866230b96d4e3"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "4fc15d21b746abcfbb94f4dc54f66781"
server_key_iv: "2aed631b3f03a642509744c3ac7dd58f"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.314520 29134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:14:54.317981 29156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:54.319793 29134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:14:54.319984 29134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal
uuid: "781b58e8aadf4eaaab3866230b96d4e3"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "4fc15d21b746abcfbb94f4dc54f66781"
server_key_iv: "2aed631b3f03a642509744c3ac7dd58f"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.320139 29134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:54.334770 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } attempt: 3
I20260807 06:14:54.336071 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.335657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53296)
0807 06:14:54.335866 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.335960 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 06:14:54.336944 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.336688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53310)
0807 06:14:54.336812 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.336869 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 06:14:54.337095 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34537: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34537: connect: Connection refused (error 111)
I20260807 06:14:54.353690 29134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:54.358186 29134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:54.358460 29134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:54.367524 29134 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.189:34537
I20260807 06:14:54.367558 29241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.189:34537 every 8 connection(s)
I20260807 06:14:54.368748 29134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
I20260807 06:14:54.372609 29243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:54.372637 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29134
I20260807 06:14:54.372831 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-1/wal/instance
I20260807 06:14:54.373205 9750 external_mini_cluster.cc:1428] Setting key 65eb770b9d6c81e591bedef67edc4dab
I20260807 06:14:54.379138 29243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.133.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.188@KRBTEST.COM" created.
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:54.397689 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.382427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.9.133.190:42203)
0807 06:14:54.382636 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.382641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.383843 (+ 1202us) server_negotiation.cc:407] Connection header received
0807 06:14:54.384997 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.385004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.385096 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.385229 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.386759 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.387431 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.388826 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.389351 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.389503 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.392849 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.392864 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.392871 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.392916 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.394683 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.395384 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.395392 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.395393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.395519 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.396069 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.396072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.396073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.396294 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.396395 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.397379 (+ 984us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.397525 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 06:14:54.399423 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.382922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42203 (local address 127.0.0.1:42884)
0807 06:14:54.383626 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.383688 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.384718 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.385426 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.385433 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.385893 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.386534 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.386550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.387586 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.387589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.388687 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.388697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.389560 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.389564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.389771 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.390599 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.390621 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.392610 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.394830 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.394838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.394869 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.395262 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.395658 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.395664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.395670 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.395943 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.396424 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.396436 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.396957 (+ 521us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.397259 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.397796 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":197,"threads_started":1}
I20260807 06:14:54.401520 29243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab.
Entry for principal kudu/127.9.133.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab.
I20260807 06:14:54.411811 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:35939
--webserver_interface=127.9.133.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab
--principal=kudu/127.9.133.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42203,127.9.133.189:34537,127.9.133.188:35939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:14:54.413300 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.402249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53316)
0807 06:14:54.402443 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.402463 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.402570 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.403566 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.403568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.403747 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.403880 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.403886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.405082 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.405085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.405709 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.405718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.406516 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.406519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.406607 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.407294 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.407308 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.408980 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.411459 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.411464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.411466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.411677 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.412056 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.412060 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.412063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.412145 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.412680 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.412683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.412780 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.412907 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.413069 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 06:14:54.413327 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.402505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.9.133.189:34537)
0807 06:14:54.402980 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.402993 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.403072 (+ 79us) server_negotiation.cc:407] Connection header received
0807 06:14:54.403121 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.403127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.403365 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.403561 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.404035 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.404903 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.405845 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.406390 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.406486 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.409133 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.409149 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.409176 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.409206 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.411325 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.411817 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.411823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.411824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.411936 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.412247 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.412254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.412256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.412553 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.412733 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.413018 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.413170 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":209,"threads_started":1}
I20260807 06:14:54.415966 29243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:14:54.416927 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.416542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54606)
0807 06:14:54.416709 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.416814 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 06:14:54.417196 29159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
I20260807 06:14:54.418319 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.417997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54616)
0807 06:14:54.418159 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.418230 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 06:14:54.418584 29243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:35939: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
I20260807 06:14:54.467923 29243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } attempt: 1
I20260807 06:14:54.469251 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.468781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54622)
0807 06:14:54.469021 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.469118 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 06:14:54.470247 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.469962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54624)
0807 06:14:54.470103 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.470168 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 06:14:54.470417 29243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:35939: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
I20260807 06:14:54.480274 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } attempt: 4
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
I20260807 06:14:54.492269 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.481055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53326)
0807 06:14:54.481273 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.481293 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.481510 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.481858 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.481861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.482091 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.482679 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.482688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.484616 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.484619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.485251 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.485265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.485898 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.485901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.486003 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.486642 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.486655 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.488436 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.490534 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.490537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.490539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.490765 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.491108 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.491111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.491112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.491203 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.491685 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.491689 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.491811 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.491922 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.492096 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 06:14:54.492309 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.481113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.9.133.189:34537)
0807 06:14:54.481277 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.481282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.481398 (+ 116us) server_negotiation.cc:407] Connection header received
0807 06:14:54.481638 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.481642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.481715 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.481814 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.482820 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.484477 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.485394 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.485774 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.485893 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.488596 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.488600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.488603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.488635 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.490416 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.490868 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.490874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.490876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.490985 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.491293 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.491296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.491297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.491566 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.491679 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.491998 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.492141 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:14:54.493407 29121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:14:54.494277 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.494006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54638)
0807 06:14:54.494142 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.494208 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 06:14:54.495069 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.494804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54654)
0807 06:14:54.494912 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.494966 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 06:14:54.495283 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:35939: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
I20260807 06:14:54.538677 29243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } attempt: 2
I20260807 06:14:54.540079 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.539588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54662)
0807 06:14:54.539828 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.539934 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 06:14:54.541215 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.540905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54678)
0807 06:14:54.541056 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.541123 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 06:14:54.541436 29243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:35939: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
I20260807 06:14:54.554474 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } attempt: 1
I20260807 06:14:54.555634 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.555257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54690)
0807 06:14:54.555448 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.555526 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 06:14:54.555941 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:54.556376 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:54.556476 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 06:14:54.556583 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.556281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54700)
0807 06:14:54.556416 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.556486 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 06:14:54.556775 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:35939: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
W20260807 06:14:54.561093 29250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:54.561157 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:54.561177 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:54.561195 29250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:54.561241 29250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:54.567472 29250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42203,127.9.133.189:34537,127.9.133.188:35939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab
--principal=kudu/127.9.133.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:35939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.133.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:54.568948 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:54.570277 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:54.580426 29260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:54.580431 29262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:54.580700 29250 server_base.cc:1034] running on GCE node
W20260807 06:14:54.580432 29259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:54.581612 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:54.583069 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:54.584246 29250 hybrid_clock.cc:648] HybridClock initialized: now 1786083294584224 us; error 40 us; skew 500 ppm
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:54.587879 29250 init.cc:377] Logged in from keytab as kudu/127.9.133.188@KRBTEST.COM (short username kudu)
I20260807 06:14:54.589069 29250 webserver.cc:460] Webserver started at http://127.9.133.188:37571/ using document root <none> and password file <none>
I20260807 06:14:54.589787 29250 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:54.589864 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:54.590150 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:54.592777 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data/instance:
uuid: "8c2b2231bac940c88dfc37b7eded399e"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "f362701b299d4313ec1b59a271300183"
server_key_iv: "c16df6b6f2d23e7084ca23a55619e1bb"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.593509 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal/instance:
uuid: "8c2b2231bac940c88dfc37b7eded399e"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "f362701b299d4313ec1b59a271300183"
server_key_iv: "c16df6b6f2d23e7084ca23a55619e1bb"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.598174 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:14:54.601281 29274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:54.602874 29250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:14:54.603129 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal
uuid: "8c2b2231bac940c88dfc37b7eded399e"
format_stamp: "Formatted at 2026-08-07 06:14:54 on dist-test-slave-h3ft"
server_key: "f362701b299d4313ec1b59a271300183"
server_key_iv: "c16df6b6f2d23e7084ca23a55619e1bb"
server_key_version: "encryptionkey@0"
I20260807 06:14:54.603258 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:54.618335 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:54.622267 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:54.622496 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:54.623965 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } attempt: 2
I20260807 06:14:54.625161 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.624771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54710)
0807 06:14:54.624955 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.625044 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 06:14:54.626129 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.625865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54716)
0807 06:14:54.626001 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.626069 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 06:14:54.626315 29121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:35939: Network error: Client connection negotiation failed: client connection to 127.9.133.188:35939: connect: Connection refused (error 111)
I20260807 06:14:54.631692 29250 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.188:35939
I20260807 06:14:54.631719 29352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.188:35939 every 8 connection(s)
I20260807 06:14:54.632658 29243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } attempt: 3
I20260807 06:14:54.633026 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
I20260807 06:14:54.637465 29353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:54.642434 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29250
I20260807 06:14:54.642612 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786083100784614-9750-0/minicluster-data/master-2/wal/instance
I20260807 06:14:54.643066 9750 external_mini_cluster.cc:1428] Setting key d9485a3103b76939c63173885b1a2ba9
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:14:54.643615 29353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:14:54.649439 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.633321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54724)
0807 06:14:54.633517 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.633537 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.633684 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.636956 (+ 3272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.636960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.637204 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.637345 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.637352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.639526 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.639530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.640123 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.640134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.641284 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.641288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.641366 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.642050 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.642065 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.644035 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.647058 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.647063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.647065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.647381 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.647826 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.647830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.647833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.647938 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.648714 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.648717 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.648862 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.648990 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.649168 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 06:14:54.651156 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.634259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.9.133.188:35939)
0807 06:14:54.634766 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.634802 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.634859 (+ 57us) server_negotiation.cc:407] Connection header received
0807 06:14:54.635967 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.635997 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.636535 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.636985 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.637551 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.639338 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.640258 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.641102 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.641302 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.644308 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.644319 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.644343 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.644383 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.646879 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.647527 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.647534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.647541 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.647703 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.648056 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.648061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.648063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.648588 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.648758 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.649178 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.649831 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":126,"threads_started":1}
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:54.657541 29243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3: Bootstrap starting.
I20260807 06:14:54.658792 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.643969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.9.133.188:35939)
0807 06:14:54.644524 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.644534 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.645210 (+ 676us) server_negotiation.cc:407] Connection header received
0807 06:14:54.645245 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.645272 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.645344 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.645453 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.646737 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.647694 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.648901 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.649274 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.650162 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.653093 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.653097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.653099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.653126 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.655406 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.655890 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.655896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.655899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.656443 (+ 544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.657460 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.657464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.657466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.657736 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.657938 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.658402 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.658542 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":229,"threads_started":1}
I20260807 06:14:54.659591 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.646450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.9.133.190:42203)
0807 06:14:54.646646 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.646651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.647475 (+ 824us) server_negotiation.cc:407] Connection header received
0807 06:14:54.647654 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.647658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.647722 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.647815 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.648823 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.649652 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.650943 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.651431 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.651573 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.655465 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.655470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.655473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.655516 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.657273 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.657845 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.657849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.657851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.657954 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.658278 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.658282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.658284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.658571 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.658674 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.659186 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.659378 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:14:54.660238 29360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.646294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:42203 (local address 127.0.0.1:42894)
0807 06:14:54.647323 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.647372 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.647540 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.647979 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.647985 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.648476 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.648648 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.648656 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.649799 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.649802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.650756 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.650767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.651751 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.651755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.651879 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.652474 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.652490 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.655281 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.657401 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.657410 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.657415 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.657683 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.658053 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.658058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.658060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.658157 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.658737 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.658744 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.658889 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.659831 (+ 942us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.660054 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":168,"threads_started":1}
I20260807 06:14:54.661785 29243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:54.662604 29353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:14:54.663753 29243 log.cc:824] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:54.666878 29243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3: No bootstrap required, opened a new log
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
I20260807 06:14:54.671399 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.660221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.9.133.190:42203)
0807 06:14:54.660346 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.660351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.660378 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:14:54.660696 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.660699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.660741 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.660831 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.661653 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.662543 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.663813 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.664203 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.664314 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.667242 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.667246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.667249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.667318 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.669275 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.669817 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.669821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.669823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.669920 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.670284 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.670287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.670289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.670541 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.670811 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.671054 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.671188 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:14:54.672304 29243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.672618 29243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:54.672680 29243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781b58e8aadf4eaaab3866230b96d4e3, State: Initialized, Role: FOLLOWER
I20260807 06:14:54.673396 29243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.673907 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.663314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.9.133.189:34537)
0807 06:14:54.663467 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.663472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.663679 (+ 207us) server_negotiation.cc:407] Connection header received
0807 06:14:54.663850 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.663854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.663912 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.664005 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.664547 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.665232 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.666374 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.666706 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.666813 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.669366 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.669371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.669374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.669414 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.671404 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.672064 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.672200 (+ 136us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.672202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.672303 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.672714 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.672718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.672720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.673080 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.673194 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.673604 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.673742 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:14:54.673911 29360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.663210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34537 (local address 127.0.0.1:53334)
0807 06:14:54.663582 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.663600 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.663694 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.664045 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.664048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.664249 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.664408 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.664416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.665369 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.665373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.666213 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.666225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.666950 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.666957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.667095 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.667706 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.667721 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.669209 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.671659 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.671663 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.671665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.671944 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.672440 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.672444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.672446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.672559 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.673258 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.673265 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.673391 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.673538 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.673704 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280}
I20260807 06:14:54.674738 29353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:14:54.675354 29243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:14:54.683298 29364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.683480 29364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.683589 29364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:14:54.683717 29364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
W20260807 06:14:54.684206 29383 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:14:54.684299 29383 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:14:54.680516 29134 master_runner.cc:186] Error getting master registration for 127.9.133.190:42203: OK, instance_id { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" instance_seqno: 1786083294082945 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:14:54.684759 29134 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:14:54.685066 29364 leader_election.cc:288] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2eac702e31d44bc92ff98938fdf6c17 (127.9.133.190:42203), 8c2b2231bac940c88dfc37b7eded399e (127.9.133.188:35939)
I20260807 06:14:54.686615 29360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.675336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54752)
0807 06:14:54.675500 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.675516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.675605 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.675859 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.675862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.676052 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.676126 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.676132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.677131 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.677138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.678068 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.678081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.678804 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.678807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.678904 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.679475 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.679491 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.681302 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.683679 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.683682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.683684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.683970 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.684490 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.684497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.684500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.684607 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.685442 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.685445 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.685572 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.686037 (+ 465us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.686235 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 06:14:54.687218 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.675366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.9.133.188:35939)
0807 06:14:54.675488 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.675493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.675561 (+ 68us) server_negotiation.cc:407] Connection header received
0807 06:14:54.675698 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.675700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.675746 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.675847 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.676249 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.677016 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.678217 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.678576 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.678689 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.681462 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.681467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.681470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.681498 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.683493 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.684286 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.684292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.684294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.684398 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.684717 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.684720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.684722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.685103 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.685214 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.686876 (+ 1662us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.687073 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:14:54.689653 29362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.673102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.9.133.189:34537)
0807 06:14:54.673689 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.673706 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.673723 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:14:54.673762 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.673765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.673812 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.674893 (+ 1081us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.675064 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.675915 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.677065 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.677465 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.677996 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.682551 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.682555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.682558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.682590 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.686958 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.687838 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.687845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.687850 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.687964 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.688394 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.688397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.688418 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.688716 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.688977 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.689243 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.689419 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":182,"threads_started":1}
I20260807 06:14:54.691459 29353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: Bootstrap starting.
I20260807 06:14:54.693864 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:14:54.695302 29353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:54.696498 29353 log.cc:824] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:54.712030 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.697146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54766 (local address 127.9.133.188:35939)
0807 06:14:54.697329 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.697335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.698312 (+ 977us) server_negotiation.cc:407] Connection header received
0807 06:14:54.698653 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.698656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.698713 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.698771 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.700430 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.701216 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.703030 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.703386 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.703479 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.706975 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.706979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.706982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.707047 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.709875 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.710387 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.710390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.710392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.710498 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.710867 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.710871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.710873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.711148 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.711256 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.711669 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.711812 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 06:14:54.712756 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.697181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.9.133.190:42203)
0807 06:14:54.697363 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.697369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.698342 (+ 973us) server_negotiation.cc:407] Connection header received
0807 06:14:54.698543 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.698548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.698629 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.698725 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.699833 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.700800 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.702124 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.702499 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.702613 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.706378 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.706382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.706386 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.706424 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.709453 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.710067 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.710073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.710077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.710198 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.710583 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.710590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.710593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.710921 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.711154 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.712004 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.712548 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:14:54.714076 29353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: No bootstrap required, opened a new log
I20260807 06:14:54.714978 29362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.697811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.9.133.189:34537)
0807 06:14:54.697957 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.697964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.698706 (+ 742us) server_negotiation.cc:407] Connection header received
0807 06:14:54.699955 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.699959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.700021 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.700078 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.702345 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.703244 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.704982 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.705530 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.705637 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.708411 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.708416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.708419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.708457 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.710316 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.711571 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.711576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.711578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.711685 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.712474 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.712480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.712486 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.712769 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.712932 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.714593 (+ 1661us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.714757 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:14:54.719273 29353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.719609 29353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:54.719683 29353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2b2231bac940c88dfc37b7eded399e, State: Initialized, Role: FOLLOWER
I20260807 06:14:54.720283 29353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.720933 29395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.721139 29395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.722070 29353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:14:54.722896 29317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "781b58e8aadf4eaaab3866230b96d4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2b2231bac940c88dfc37b7eded399e" is_pre_election: true
I20260807 06:14:54.723367 29317 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 781b58e8aadf4eaaab3866230b96d4e3 in term 0.
I20260807 06:14:54.723935 29159 leader_election.cc:302] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781b58e8aadf4eaaab3866230b96d4e3, 8c2b2231bac940c88dfc37b7eded399e; no voters:
I20260807 06:14:54.724537 29384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:14:54.724679 29384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:14:54.724737 29384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:54.726512 29121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } attempt: 3
I20260807 06:14:54.726750 29384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.727634 29384 leader_election.cc:288] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [CANDIDATE]: Term 1 election: Requested vote from peers a2eac702e31d44bc92ff98938fdf6c17 (127.9.133.190:42203), 8c2b2231bac940c88dfc37b7eded399e (127.9.133.188:35939)
I20260807 06:14:54.728780 29317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "781b58e8aadf4eaaab3866230b96d4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2b2231bac940c88dfc37b7eded399e"
I20260807 06:14:54.728931 29317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:54.725906 29250 master_runner.cc:186] Error getting master registration for 127.9.133.190:42203: OK, instance_id { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" instance_seqno: 1786083294082945 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:14:54.729576 29250 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 06:14:54.730407 29410 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:14:54.730485 29410 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:14:54.730585 29317 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 781b58e8aadf4eaaab3866230b96d4e3 in term 1.
I20260807 06:14:54.730983 29159 leader_election.cc:302] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781b58e8aadf4eaaab3866230b96d4e3, 8c2b2231bac940c88dfc37b7eded399e; no voters:
I20260807 06:14:54.731304 29384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:14:54.731585 29384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [term 1 LEADER]: Becoming Leader. State: Replica: 781b58e8aadf4eaaab3866230b96d4e3, State: Running, Role: LEADER
I20260807 06:14:54.733517 29384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
Aug 07 06:14:54 dist-test-slave-h3ft krb5kdc[29002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:14:54.740187 29364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 781b58e8aadf4eaaab3866230b96d4e3. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.740363 29364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:54.741015 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.727482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:35939 (local address 127.0.0.1:54776)
0807 06:14:54.727847 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:54.727870 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:54.728006 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:54.730811 (+ 2805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:54.730814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:54.731204 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:54.731377 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.731387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.732515 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.732520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.733439 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:54.733452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.734208 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.734212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:54.734305 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.735048 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:54.735071 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:54.736838 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:54.739163 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.739167 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.739169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.739479 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.739796 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:54.739799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:54.739802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:54.739890 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:54.740384 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:54.740387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:54.740497 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 06:14:54.740586 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 06:14:54.740745 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257}
I20260807 06:14:54.741250 29416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:14:54.741603 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:54.727228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.9.133.188:35939)
0807 06:14:54.727421 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:54.727427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:54.728251 (+ 824us) server_negotiation.cc:407] Connection header received
0807 06:14:54.728303 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:54.728307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:54.728364 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:54.728443 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:54.731531 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.732358 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.733589 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:54.733965 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:54.734065 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:54.737012 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:54.737016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:54.737019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:54.737046 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:54.738873 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.739599 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.739603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.739605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.739696 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:54.740004 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:54.740007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:54.740009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:54.740248 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:54.740339 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:54.741301 (+ 962us) server_negotiation.cc:299] Negotiation successful
0807 06:14:54.741434 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 06:14:54.743854 29416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:14:54.745604 29121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: Bootstrap starting.
I20260807 06:14:54.749325 29121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:54.750828 29121 log.cc:824] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:54.754223 29121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: No bootstrap required, opened a new log
I20260807 06:14:54.762652 29121 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.763731 29317 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [term 1 FOLLOWER]: Refusing update from remote peer 781b58e8aadf4eaaab3866230b96d4e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:14:54.764604 29384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:14:54.769356 29121 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:54.769552 29121 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eac702e31d44bc92ff98938fdf6c17, State: Initialized, Role: FOLLOWER
I20260807 06:14:54.770259 29121 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } }
I20260807 06:14:54.770619 29395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 781b58e8aadf4eaaab3866230b96d4e3. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.770762 29395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.772207 29121 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:14:54.772511 29081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:54.774456 29081 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 1 FOLLOWER]: Refusing update from remote peer 781b58e8aadf4eaaab3866230b96d4e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:14:54.776186 29384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:14:54.774978 29083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "781b58e8aadf4eaaab3866230b96d4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2eac702e31d44bc92ff98938fdf6c17" is_pre_election: true
I20260807 06:14:54.777655 29083 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 781b58e8aadf4eaaab3866230b96d4e3 in term 1.
I20260807 06:14:54.775609 29082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "781b58e8aadf4eaaab3866230b96d4e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2eac702e31d44bc92ff98938fdf6c17"
I20260807 06:14:54.780077 29082 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 781b58e8aadf4eaaab3866230b96d4e3 in term 1.
I20260807 06:14:54.787066 29422 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.787328 29422 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.787508 29422 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 781b58e8aadf4eaaab3866230b96d4e3. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.787688 29422 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.789472 29364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.789619 29364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:54.789713 29364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.789774 29364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3 [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:54.790596 29395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.790673 29422 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "781b58e8aadf4eaaab3866230b96d4e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2eac702e31d44bc92ff98938fdf6c17" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42203 } } peers { permanent_uuid: "781b58e8aadf4eaaab3866230b96d4e3" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34537 } } peers { permanent_uuid: "8c2b2231bac940c88dfc37b7eded399e" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 35939 } } }
I20260807 06:14:54.790738 29395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.790771 29422 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:14:54.791333 29420 mvcc.cc:205] Tried to move back new op lower bound from 7315797175342833664 to 7315797175237779456. Current Snapshot: MvccSnapshot[applied={T|T < 7315797175342833664}]
I20260807 06:14:54.793956 29416 catalog_manager.cc:1347] Generated new cluster ID: 0753b75f1fac40a3b80b47c698803285
I20260807 06:14:54.794039 29416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260807 06:14:54.799393 29438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:14:54.817519 29438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:14:54.817659 29416 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:14:54.819164 29416 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:14:54.817381 29440 mvcc.cc:205] Tried to move back new op lower bound from 7315797175342833664 to 7315797175237779456. Current Snapshot: MvccSnapshot[applied={T|T < 7315797175342833664}]
I20260807 06:14:54.833891 29416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 781b58e8aadf4eaaab3866230b96d4e3: Generated new TSK 0
I20260807 06:14:54.834997 29416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:14:55.732450 29410 catalog_manager.cc:1259] Loaded cluster ID: 0753b75f1fac40a3b80b47c698803285
I20260807 06:14:55.732590 29410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: loading cluster ID for follower catalog manager: success
I20260807 06:14:55.734680 29410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: acquiring CA information for follower catalog manager: success
I20260807 06:14:55.736306 29410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c2b2231bac940c88dfc37b7eded399e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:14:55.819378 29438 catalog_manager.cc:1259] Loaded cluster ID: 0753b75f1fac40a3b80b47c698803285
I20260807 06:14:55.819555 29438 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: loading cluster ID for follower catalog manager: success
I20260807 06:14:55.821586 29438 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: acquiring CA information for follower catalog manager: success
I20260807 06:14:55.823076 29438 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2eac702e31d44bc92ff98938fdf6c17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:14:55.916847 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29023
I20260807 06:14:55.934919 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29134
I20260807 06:14:55.948848 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29250
2026-08-07T06:14:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3526 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:14:56 dist-test-slave-h3ft krb5kdc[29466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:14:56 dist-test-slave-h3ft krb5kdc[29466](info): set up 2 sockets
Aug 07 06:14:56 dist-test-slave-h3ft krb5kdc[29466](info): commencing operation
krb5kdc: starting...
W20260807 06:14:57.217710 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:14:57 dist-test-slave-h3ft krb5kdc[29466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:14:57.370854 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:33653
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:33653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:57.503733 29485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:57.504125 29485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:57.504215 29485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:57.508435 29485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:57.508554 29485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:57.508589 29485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:57.508634 29485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:57.508675 29485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:57.514549 29485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:33653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:33653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:57.516156 29485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:57.517534 29485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:57.527877 29491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:57.527877 29492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:57.527870 29494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:57.528312 29485 server_base.cc:1034] running on GCE node
I20260807 06:14:57.529006 29485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:57.530459 29485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:57.531697 29485 hybrid_clock.cc:648] HybridClock initialized: now 1786083297531682 us; error 40 us; skew 500 ppm
Aug 07 06:14:57 dist-test-slave-h3ft krb5kdc[29466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:57.540009 29485 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:14:57.541360 29485 webserver.cc:460] Webserver started at http://127.9.133.190:46727/ using document root <none> and password file <none>
I20260807 06:14:57.542127 29485 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:57.542232 29485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:57.542536 29485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:57.545200 29485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "18f433f8501b4aed83e56b2b22db1a3f"
format_stamp: "Formatted at 2026-08-07 06:14:57 on dist-test-slave-h3ft"
server_key: "2a7cff06f1cee484ef7e4555457e9444"
server_key_iv: "8e5a640a90c46f9232886c314389b354"
server_key_version: "encryptionkey@0"
I20260807 06:14:57.545851 29485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "18f433f8501b4aed83e56b2b22db1a3f"
format_stamp: "Formatted at 2026-08-07 06:14:57 on dist-test-slave-h3ft"
server_key: "2a7cff06f1cee484ef7e4555457e9444"
server_key_iv: "8e5a640a90c46f9232886c314389b354"
server_key_version: "encryptionkey@0"
I20260807 06:14:57.550482 29485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260807 06:14:57.553534 29505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:57.555151 29485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:14:57.555317 29485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "18f433f8501b4aed83e56b2b22db1a3f"
format_stamp: "Formatted at 2026-08-07 06:14:57 on dist-test-slave-h3ft"
server_key: "2a7cff06f1cee484ef7e4555457e9444"
server_key_iv: "8e5a640a90c46f9232886c314389b354"
server_key_version: "encryptionkey@0"
I20260807 06:14:57.555445 29485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:57.570923 29485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:57.574621 29485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:57.574877 29485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:57.584276 29485 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:33653
I20260807 06:14:57.584334 29585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:33653 every 8 connection(s)
I20260807 06:14:57.585712 29485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:14:57.589582 29586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:57.590565 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29485
I20260807 06:14:57.590734 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:14:57.591189 9750 external_mini_cluster.cc:1428] Setting key 0056d52cdbe4ceaec5546f7f6f54be6e
I20260807 06:14:57.597873 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f: Bootstrap starting.
I20260807 06:14:57.601420 29586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f: Neither blocks nor log segments found. Creating new log.
Aug 07 06:14:57 dist-test-slave-h3ft krb5kdc[29466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:57.602402 29586 log.cc:824] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:57.605199 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f: No bootstrap required, opened a new log
I20260807 06:14:57.607960 29589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:57.593013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.9.133.190:33653)
0807 06:14:57.593535 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:57.593555 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:57.593603 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:14:57.594347 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:57.594367 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:57.594775 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:57.595209 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:57.596261 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:57.597388 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:57.598592 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:57.599226 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:57.599362 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:57.602630 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:57.602636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:57.602653 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:57.602685 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:57.604644 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:57.605143 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:57.605148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:57.605153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:57.605274 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:57.605641 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:57.605643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:57.605644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:57.606100 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:57.606282 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:57.606652 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 06:14:57.607011 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":162,"threads_started":1}
I20260807 06:14:57.609820 29586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f433f8501b4aed83e56b2b22db1a3f" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33653 } }
I20260807 06:14:57.610128 29586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:57.610200 29586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f433f8501b4aed83e56b2b22db1a3f, State: Initialized, Role: FOLLOWER
I20260807 06:14:57.610695 29586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f433f8501b4aed83e56b2b22db1a3f" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33653 } }
I20260807 06:14:57.610885 29586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:14:57.610972 29586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:14:57.611140 29586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:57.611361 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:14:57.612589 29586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f433f8501b4aed83e56b2b22db1a3f" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33653 } }
I20260807 06:14:57.613014 29586 leader_election.cc:302] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f433f8501b4aed83e56b2b22db1a3f; no voters:
I20260807 06:14:57.613452 29586 leader_election.cc:288] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:14:57.613583 29593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:14:57.613847 29593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [term 1 LEADER]: Becoming Leader. State: Replica: 18f433f8501b4aed83e56b2b22db1a3f, State: Running, Role: LEADER
I20260807 06:14:57.614207 29593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f433f8501b4aed83e56b2b22db1a3f" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33653 } }
I20260807 06:14:57.615058 29586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:14:57.616262 29594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f433f8501b4aed83e56b2b22db1a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f433f8501b4aed83e56b2b22db1a3f" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33653 } } }
I20260807 06:14:57.616407 29594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:57.616729 29596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f433f8501b4aed83e56b2b22db1a3f. Latest consensus state: current_term: 1 leader_uuid: "18f433f8501b4aed83e56b2b22db1a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f433f8501b4aed83e56b2b22db1a3f" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33653 } } }
I20260807 06:14:57.616822 29596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:57.616848 29608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:14:57.620253 29608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:14:57.627182 29608 catalog_manager.cc:1347] Generated new cluster ID: 2f38abc3c47a469c97c205b451e98137
I20260807 06:14:57.627282 29608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:14:57.652534 29608 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:14:57.654218 29608 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:14:57.671360 29608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f433f8501b4aed83e56b2b22db1a3f: Generated new TSK 0
I20260807 06:14:57.672565 29608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:14:57.720737 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29485
2026-08-07T06:14:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1773 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:14:57 dist-test-slave-h3ft krb5kdc[29636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:14:57 dist-test-slave-h3ft krb5kdc[29636](info): set up 2 sockets
Aug 07 06:14:57 dist-test-slave-h3ft krb5kdc[29636](info): commencing operation
krb5kdc: starting...
W20260807 06:14:59.008737 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:14:59 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:14:59.163196 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:32927
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:59.296525 29656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:59.296882 29656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:59.296926 29656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:59.301173 29656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:14:59.301244 29656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:59.301263 29656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:59.301281 29656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:14:59.301296 29656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:14:59.307086 29656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:59.308499 29656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:59.309865 29656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:59.320451 29666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:59.320510 29663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:59.320529 29664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:59.320662 29656 server_base.cc:1034] running on GCE node
I20260807 06:14:59.321466 29656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:59.322906 29656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:59.324083 29656 hybrid_clock.cc:648] HybridClock initialized: now 1786083299324055 us; error 39 us; skew 500 ppm
Aug 07 06:14:59 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:59.327514 29656 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:14:59.328706 29656 webserver.cc:460] Webserver started at http://127.9.133.190:38847/ using document root <none> and password file <none>
I20260807 06:14:59.329449 29656 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:59.329522 29656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:59.329818 29656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:59.332463 29656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "161e481d0e184eb8bf6bbfc6d27cf30c"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "a3338763e4cc59791df04b499922a4d6"
server_key_iv: "25e2e46372153ef019f1f0d147154c6e"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.333164 29656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "161e481d0e184eb8bf6bbfc6d27cf30c"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "a3338763e4cc59791df04b499922a4d6"
server_key_iv: "25e2e46372153ef019f1f0d147154c6e"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.337785 29656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:14:59.340888 29675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:59.342597 29656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:14:59.342783 29656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "161e481d0e184eb8bf6bbfc6d27cf30c"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "a3338763e4cc59791df04b499922a4d6"
server_key_iv: "25e2e46372153ef019f1f0d147154c6e"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.342970 29656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:59.366492 29656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:59.370559 29656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:59.370843 29656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:59.380260 29656 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:32927
I20260807 06:14:59.380630 29749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:32927 every 8 connection(s)
I20260807 06:14:59.381575 29656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:14:59.382642 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29656
I20260807 06:14:59.382824 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:14:59.383232 9750 external_mini_cluster.cc:1428] Setting key 8919ad49cee6735337da6163b3088efc
I20260807 06:14:59.385783 29750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:14:59.394203 29750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap starting.
Aug 07 06:14:59 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:59.397262 29750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Neither blocks nor log segments found. Creating new log.
I20260807 06:14:59.398175 29750 log.cc:824] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:14:59.400794 29750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: No bootstrap required, opened a new log
I20260807 06:14:59.401561 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:59.385177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.9.133.190:32927)
0807 06:14:59.385703 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:59.385722 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:59.385766 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:14:59.386632 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:59.386653 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:59.387133 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:59.387488 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:59.388560 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:59.389714 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:59.390982 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:59.391657 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:59.391808 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:59.395417 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:59.395424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:59.395441 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:59.395482 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:59.397742 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:59.398328 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:59.398337 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:59.398345 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:59.398498 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:59.398835 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:59.398840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:59.398842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:59.399487 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:59.399754 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:59.400111 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 06:14:59.400499 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":166,"threads_started":1}
I20260807 06:14:59.405617 29750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:14:59.405925 29750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:14:59.406016 29750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161e481d0e184eb8bf6bbfc6d27cf30c, State: Initialized, Role: FOLLOWER
I20260807 06:14:59.406569 29750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:14:59.406741 29750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:14:59.406834 29750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:14:59.406957 29750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:14:59.408265 29750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:14:59.408694 29750 leader_election.cc:302] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161e481d0e184eb8bf6bbfc6d27cf30c; no voters:
I20260807 06:14:59.409135 29750 leader_election.cc:288] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:14:59.409303 29757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:14:59.409606 29757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 LEADER]: Becoming Leader. State: Replica: 161e481d0e184eb8bf6bbfc6d27cf30c, State: Running, Role: LEADER
I20260807 06:14:59.409965 29757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:14:59.410750 29750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:14:59.411995 29760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161e481d0e184eb8bf6bbfc6d27cf30c. Latest consensus state: current_term: 1 leader_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } } }
I20260807 06:14:59.412149 29760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:59.412480 29758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } } }
I20260807 06:14:59.412567 29768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:14:59.412648 29758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: This master's current role is: LEADER
I20260807 06:14:59.416594 29768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:14:59.423591 29768 catalog_manager.cc:1347] Generated new cluster ID: bb674bb300694cc0a60d67fb2223968e
I20260807 06:14:59.423753 29768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:14:59.459846 29768 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:14:59.461139 29768 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:14:59.470595 29768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Generated new TSK 0
I20260807 06:14:59.471675 29768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:14:59.515241 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:32927
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:59.651235 29785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:59.651671 29785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:59.651759 29785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:59.656323 29785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:59.656461 29785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:59.656574 29785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:14:59.662688 29785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:59.664219 29785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:59.665611 29785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:59.677256 29801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:59.677225 29795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:59.677229 29791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:59.677847 29785 server_base.cc:1034] running on GCE node
I20260807 06:14:59.678377 29785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:59.679271 29785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:59.680473 29785 hybrid_clock.cc:648] HybridClock initialized: now 1786083299680453 us; error 31 us; skew 500 ppm
Aug 07 06:14:59 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:59.683832 29785 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:14:59.684991 29785 webserver.cc:460] Webserver started at http://127.9.133.129:36001/ using document root <none> and password file <none>
I20260807 06:14:59.685709 29785 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:59.685806 29785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:59.686095 29785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:59.688685 29785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "4f51469ae9114174ad5b714981331150"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "b6c12924bd1adfefa263b762bf845d5c"
server_key_iv: "d29a54d4b315c1e44f275d6701c68b46"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.689368 29785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "4f51469ae9114174ad5b714981331150"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "b6c12924bd1adfefa263b762bf845d5c"
server_key_iv: "d29a54d4b315c1e44f275d6701c68b46"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.694056 29785 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:14:59.697077 29810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:59.698701 29785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 06:14:59.698879 29785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "4f51469ae9114174ad5b714981331150"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "b6c12924bd1adfefa263b762bf845d5c"
server_key_iv: "d29a54d4b315c1e44f275d6701c68b46"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.699067 29785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:14:59.728523 29785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:14:59.733158 29785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:14:59.733412 29785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:14:59.734185 29785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:14:59.735287 29785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:14:59.735338 29785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:59.735384 29785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:14:59.735399 29785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:59.745988 29785 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:42921
I20260807 06:14:59.746026 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:42921 every 8 connection(s)
I20260807 06:14:59.747212 29785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:14:59.755995 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29785
I20260807 06:14:59.756175 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:14:59.756542 9750 external_mini_cluster.cc:1428] Setting key 9ceb030e9730f5c588499d4895ae7776
Aug 07 06:14:59 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:14:59.764122 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:59.749786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:38533 (local address 127.9.133.190:32927)
0807 06:14:59.749965 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 06:14:59.749969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:14:59.751229 (+ 1260us) server_negotiation.cc:407] Connection header received
0807 06:14:59.752297 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:14:59.752300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:14:59.752363 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:14:59.752444 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:14:59.753912 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:59.754607 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:59.756173 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:14:59.756504 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:14:59.756626 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:59.759695 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:14:59.759698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:14:59.759701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:14:59.759727 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:14:59.761412 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:59.762061 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:59.762064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:59.762065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:59.762152 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:14:59.762575 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:14:59.762581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:14:59.762586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:14:59.762855 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:14:59.762974 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 06:14:59.763684 (+ 710us) server_negotiation.cc:299] Negotiation successful
0807 06:14:59.763849 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:14:59.765043 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:14:59.750206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:38533)
0807 06:14:59.751072 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 06:14:59.751126 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:14:59.752054 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:14:59.752659 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:14:59.752666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:14:59.753089 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:14:59.753681 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:59.753728 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:59.754762 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:59.754765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:59.756019 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:14:59.756034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:14:59.756698 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:14:59.756703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:14:59.756913 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:14:59.757689 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:14:59.757707 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:14:59.759511 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:14:59.761534 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:59.761543 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:59.761563 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:59.761952 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:59.762239 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:14:59.762244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:14:59.762247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:14:59.762457 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:14:59.763081 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:14:59.763088 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:14:59.763370 (+ 282us) client_negotiation.cc:769] Sending connection context
0807 06:14:59.763575 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 06:14:59.763873 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":135,"threads_started":1}
I20260807 06:14:59.766741 29984 heartbeater.cc:347] Connected to a master server at 127.9.133.190:32927
I20260807 06:14:59.767079 29984 heartbeater.cc:464] Registering TS with master...
I20260807 06:14:59.767696 29984 heartbeater.cc:511] Master 127.9.133.190:32927 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:14:59.769682 29693 ts_manager.cc:194] Registered new tserver with Master: 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921)
I20260807 06:14:59.771200 29693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:38533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:14:59.794512 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:32927
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:14:59.934000 29990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:14:59.934363 29990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:14:59.934448 29990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:14:59.938707 29990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:14:59.938822 29990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:14:59.938967 29990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:14:59.944734 29990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:14:59.946141 29990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:14:59.947600 29990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:14:59.959322 29996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:59.959342 30000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:14:59.959302 29995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:14:59.959864 29990 server_base.cc:1034] running on GCE node
I20260807 06:14:59.960330 29990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:14:59.961143 29990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:14:59.962338 29990 hybrid_clock.cc:648] HybridClock initialized: now 1786083299962321 us; error 34 us; skew 500 ppm
Aug 07 06:14:59 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:14:59.966022 29990 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:14:59.967319 29990 webserver.cc:460] Webserver started at http://127.9.133.130:35999/ using document root <none> and password file <none>
I20260807 06:14:59.968010 29990 fs_manager.cc:359] Metadata directory not provided
I20260807 06:14:59.968077 29990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:14:59.968292 29990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:14:59.970984 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "d953a446c00e7483336812b9a8b7e3da"
server_key_iv: "5d3de85efb17a33ddfa494848bd06493"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.971715 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "d953a446c00e7483336812b9a8b7e3da"
server_key_iv: "5d3de85efb17a33ddfa494848bd06493"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.976675 29990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 06:14:59.980178 30012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:14:59.981829 29990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:14:59.982012 29990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "d953a446c00e7483336812b9a8b7e3da"
server_key_iv: "5d3de85efb17a33ddfa494848bd06493"
server_key_version: "encryptionkey@0"
I20260807 06:14:59.982163 29990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:00.000741 29990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:00.005375 29990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:00.005661 29990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:00.006534 29990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:00.007754 29990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:00.007813 29990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:00.007903 29990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:00.007941 29990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:00.020711 29990 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:33645
I20260807 06:15:00.020735 30173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:33645 every 8 connection(s)
I20260807 06:15:00.021988 29990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:15:00.024689 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 29990
I20260807 06:15:00.024847 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:15:00.025207 9750 external_mini_cluster.cc:1428] Setting key f3798e6cea245ea919423893829dc9f0
Aug 07 06:15:00 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083299, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:15:00.041508 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.024638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34013 (local address 127.9.133.190:32927)
0807 06:15:00.024843 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.024849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.026375 (+ 1526us) server_negotiation.cc:407] Connection header received
0807 06:15:00.027663 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.027666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.027725 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.027809 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:00.029894 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.030853 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.032271 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.032700 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.032838 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.036375 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:00.036383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:00.036389 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:00.036437 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:00.038573 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:00.039399 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:00.039406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:00.039411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:00.039531 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:00.040043 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:00.040049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:00.040053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:00.040318 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:00.040408 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.041155 (+ 747us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.041307 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 06:15:00.042769 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.025168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:34013)
0807 06:15:00.026150 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.026222 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.027372 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.028094 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.028108 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.028765 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:00.029682 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.029709 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.031074 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.031083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.032121 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.032131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.032892 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.032895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.033082 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.033874 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:00.033893 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:00.036136 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:00.038743 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:00.038749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:00.038768 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:00.039257 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:00.039664 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:00.039667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:00.039668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:00.039915 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:00.040450 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:00.040457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:00.040817 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.041056 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.041515 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":142,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:15:00.044481 30174 heartbeater.cc:347] Connected to a master server at 127.9.133.190:32927
I20260807 06:15:00.044836 30174 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:00.045629 30174 heartbeater.cc:511] Master 127.9.133.190:32927 requested a full tablet report, sending...
I20260807 06:15:00.047315 29693 ts_manager.cc:194] Registered new tserver with Master: 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645)
I20260807 06:15:00.048273 29693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:34013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:15:00.064705 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:32927
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:00.204319 30186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:00.204668 30186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:00.204717 30186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:00.208992 30186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:00.209076 30186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:00.209173 30186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:15:00.215549 30186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:00.217072 30186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:00.218438 30186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:00.230311 30197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:00.230311 30194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:00.230311 30195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:00.230914 30186 server_base.cc:1034] running on GCE node
I20260807 06:15:00.231417 30186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:00.232226 30186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:00.233433 30186 hybrid_clock.cc:648] HybridClock initialized: now 1786083300233414 us; error 32 us; skew 500 ppm
Aug 07 06:15:00 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:00.236739 30186 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:15:00.237746 30186 webserver.cc:460] Webserver started at http://127.9.133.131:38079/ using document root <none> and password file <none>
I20260807 06:15:00.238400 30186 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:00.238461 30186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:00.238660 30186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:15:00.241206 30186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "8584d84f032d4005903758430c8cff03"
format_stamp: "Formatted at 2026-08-07 06:15:00 on dist-test-slave-h3ft"
server_key: "4c4b034b3bb8a81c0dd6aec9e0ba1ca9"
server_key_iv: "6f98fa5f082ae2a239ea62b188c55d93"
server_key_version: "encryptionkey@0"
I20260807 06:15:00.241816 30186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "8584d84f032d4005903758430c8cff03"
format_stamp: "Formatted at 2026-08-07 06:15:00 on dist-test-slave-h3ft"
server_key: "4c4b034b3bb8a81c0dd6aec9e0ba1ca9"
server_key_iv: "6f98fa5f082ae2a239ea62b188c55d93"
server_key_version: "encryptionkey@0"
I20260807 06:15:00.246335 30186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:15:00.249383 30207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:00.251192 30186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 06:15:00.251317 30186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "8584d84f032d4005903758430c8cff03"
format_stamp: "Formatted at 2026-08-07 06:15:00 on dist-test-slave-h3ft"
server_key: "4c4b034b3bb8a81c0dd6aec9e0ba1ca9"
server_key_iv: "6f98fa5f082ae2a239ea62b188c55d93"
server_key_version: "encryptionkey@0"
I20260807 06:15:00.251432 30186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:00.265152 30186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:00.269243 30186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:00.269455 30186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:00.270180 30186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:00.271273 30186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:00.271333 30186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:00.271381 30186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:00.271399 30186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:00.282085 30186 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:36105
I20260807 06:15:00.282107 30369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:36105 every 8 connection(s)
I20260807 06:15:00.283306 30186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:15:00.285203 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 30186
I20260807 06:15:00.285368 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:15:00.285861 9750 external_mini_cluster.cc:1428] Setting key 666129611192823627fc84e3ca903683
Aug 07 06:15:00 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:00.300040 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.285895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:43911 (local address 127.9.133.190:32927)
0807 06:15:00.286105 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.286111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.287697 (+ 1586us) server_negotiation.cc:407] Connection header received
0807 06:15:00.288883 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.288886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.288951 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.289022 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:00.290528 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.291247 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.292552 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.292813 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.292906 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.296158 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:00.296162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:00.296164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:00.296196 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:00.297613 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:00.298142 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:00.298144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:00.298146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:00.298233 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:00.298578 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:00.298581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:00.298582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:00.298803 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:00.298888 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.299667 (+ 779us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.299831 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 06:15:00.301366 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.286404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:43911)
0807 06:15:00.287494 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.287568 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.288615 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.289208 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.289216 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.289687 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:00.290337 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.290356 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.291388 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.291392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.292423 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.292434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.292967 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.292994 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.293229 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.294001 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:00.294020 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:00.295965 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:00.297726 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:00.297732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:00.297747 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:00.298051 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:00.298314 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:00.298317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:00.298318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:00.298480 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:00.298947 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:00.298954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:00.299371 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.299624 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.300042 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":175,"threads_started":1}
I20260807 06:15:00.302673 30370 heartbeater.cc:347] Connected to a master server at 127.9.133.190:32927
I20260807 06:15:00.302950 30370 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:00.303669 30370 heartbeater.cc:511] Master 127.9.133.190:32927 requested a full tablet report, sending...
I20260807 06:15:00.304996 29693 ts_manager.cc:194] Registered new tserver with Master: 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
I20260807 06:15:00.305727 29693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:43911
I20260807 06:15:00.313730 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:15:00.324712 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.316299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57102 (local address 127.9.133.190:32927)
0807 06:15:00.316462 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.316466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.316765 (+ 299us) server_negotiation.cc:407] Connection header received
0807 06:15:00.316890 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.316894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.316944 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.317045 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:00.317809 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.318497 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.319508 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.319739 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.319819 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.321341 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:00.321345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:00.321347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:00.321375 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:00.323042 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:00.323487 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:00.323490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:00.323491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:00.323574 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:00.323811 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:00.323813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:00.323815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:00.324104 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:00.324220 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.324433 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.324548 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 06:15:00.328506 29693 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:15:00.332418 29693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:15:00.350754 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.343408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33645 (local address 127.0.0.1:59970)
0807 06:15:00.343898 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.343961 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.344114 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.345802 (+ 1688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.345808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.345833 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.346036 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.346044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.347706 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.347710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.348892 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.348902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.350048 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.350052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.350212 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.350265 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.350389 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.350550 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":41,"thread_start_us":130,"threads_started":1}
I20260807 06:15:00.350752 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.344428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.9.133.130:33645)
0807 06:15:00.345218 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.345234 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.345268 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:15:00.345376 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.345383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.345623 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.345909 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.346138 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.347528 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.349019 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.349890 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.350188 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.350324 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.350418 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.350557 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":162,"threads_started":1}
I20260807 06:15:00.350806 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.344412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.9.133.129:42921)
0807 06:15:00.345443 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.345450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.345470 (+ 20us) server_negotiation.cc:407] Connection header received
0807 06:15:00.345548 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.345552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.345751 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.345909 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.346241 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.347717 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.349279 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.350072 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.350172 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.350324 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.350546 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.350652 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":166,"threads_started":1}
I20260807 06:15:00.351220 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.343777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:36105 (local address 127.0.0.1:44224)
0807 06:15:00.344468 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.344483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.344600 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.345761 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.345767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.345793 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.346220 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.346233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.347480 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.347487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.349142 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.349153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.350028 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.350032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.350632 (+ 600us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.350655 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.350721 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.350796 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":132,"spinlock_wait_cycles":3840,"thread_start_us":114,"threads_started":1}
I20260807 06:15:00.351605 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.343470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42921 (local address 127.0.0.1:51126)
0807 06:15:00.344452 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.344469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.344581 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.345997 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.346000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.346015 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.346108 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.346118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.347800 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.347804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.349142 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.349153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.350310 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.350313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.350410 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.350431 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.350498 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.350551 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":65,"thread_start_us":162,"threads_started":1}
I20260807 06:15:00.351601 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.344838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.9.133.131:36105)
0807 06:15:00.345258 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.345271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.345297 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:15:00.345385 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.345389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.345585 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.345777 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.346356 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.347284 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.349319 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.349885 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.350032 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.350232 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.351318 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.351443 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":212,"threads_started":1}
I20260807 06:15:00.354923 29886 tablet_service.cc:1467] Processing CreateTablet for tablet fedd847a851647e8a452c92f4fefa121 (DEFAULT_TABLE table=test-table [id=6c932937608c4cfd85ab625b786de499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:00.354923 30277 tablet_service.cc:1467] Processing CreateTablet for tablet fedd847a851647e8a452c92f4fefa121 (DEFAULT_TABLE table=test-table [id=6c932937608c4cfd85ab625b786de499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:00.354930 30081 tablet_service.cc:1467] Processing CreateTablet for tablet fedd847a851647e8a452c92f4fefa121 (DEFAULT_TABLE table=test-table [id=6c932937608c4cfd85ab625b786de499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:00.356746 30277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fedd847a851647e8a452c92f4fefa121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:00.356739 29886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fedd847a851647e8a452c92f4fefa121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:00.356746 30081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fedd847a851647e8a452c92f4fefa121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:00.365217 30392 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Bootstrap starting.
I20260807 06:15:00.366184 30393 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Bootstrap starting.
I20260807 06:15:00.367274 30394 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Bootstrap starting.
I20260807 06:15:00.369171 30392 tablet_bootstrap.cc:654] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:00.369171 30393 tablet_bootstrap.cc:654] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:00.369801 30394 tablet_bootstrap.cc:654] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:00.370400 30392 log.cc:824] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:00.370559 30393 log.cc:824] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:00.370606 30394 log.cc:824] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:00.372874 30394 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: No bootstrap required, opened a new log
I20260807 06:15:00.372879 30392 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: No bootstrap required, opened a new log
I20260807 06:15:00.373143 30392 ts_tablet_manager.cc:1397] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:15:00.373154 30394 ts_tablet_manager.cc:1397] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 06:15:00.374562 30393 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: No bootstrap required, opened a new log
I20260807 06:15:00.374768 30393 ts_tablet_manager.cc:1397] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 06:15:00.377614 30392 raft_consensus.cc:348] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.377931 30392 raft_consensus.cc:374] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:00.377991 30392 raft_consensus.cc:729] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51469ae9114174ad5b714981331150, State: Initialized, Role: FOLLOWER
I20260807 06:15:00.377849 30394 raft_consensus.cc:348] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.378152 30394 raft_consensus.cc:374] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:00.378218 30394 raft_consensus.cc:729] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9980c3aa9b8a4ace816a532c0ffa8800, State: Initialized, Role: FOLLOWER
I20260807 06:15:00.378655 30392 consensus_queue.cc:260] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.378681 30394 consensus_queue.cc:260] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.378939 30393 raft_consensus.cc:348] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.379278 30393 raft_consensus.cc:374] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:00.379349 30393 raft_consensus.cc:729] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8584d84f032d4005903758430c8cff03, State: Initialized, Role: FOLLOWER
I20260807 06:15:00.379962 29984 heartbeater.cc:503] Master 127.9.133.190:32927 was elected leader, sending a full tablet report...
I20260807 06:15:00.380038 30394 ts_tablet_manager.cc:1428] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 06:15:00.380133 30392 ts_tablet_manager.cc:1428] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260807 06:15:00.380571 30174 heartbeater.cc:503] Master 127.9.133.190:32927 was elected leader, sending a full tablet report...
I20260807 06:15:00.379792 30393 consensus_queue.cc:260] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.382639 30393 ts_tablet_manager.cc:1428] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:15:00.383157 30370 heartbeater.cc:503] Master 127.9.133.190:32927 was elected leader, sending a full tablet report...
W20260807 06:15:00.498929 29985 tablet.cc:2369] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:15:00.523504 30175 tablet.cc:2369] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:15:00.534598 30371 tablet.cc:2369] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:15:00.605597 30407 raft_consensus.cc:482] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:00.605919 30407 raft_consensus.cc:504] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.606806 30402 raft_consensus.cc:482] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:00.607087 30402 raft_consensus.cc:504] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.607805 30407 leader_election.cc:288] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921)
I20260807 06:15:00.609321 30402 leader_election.cc:288] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
I20260807 06:15:00.613353 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.608140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42921 (local address 127.9.133.131:52551)
0807 06:15:00.608318 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.608339 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.608487 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.608961 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.608965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.608991 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.609148 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.609165 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.610324 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.610328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.611766 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.611773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.612836 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.612840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.612979 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.613007 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.613076 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.613165 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":41}
I20260807 06:15:00.613795 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.608267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:52551 (local address 127.9.133.129:42921)
0807 06:15:00.608428 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.608432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.608451 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:15:00.608647 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.608651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.608724 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.608804 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.609343 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.610192 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.611940 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.612562 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.612702 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.612770 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.613513 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.613620 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:15:00.614344 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.608326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:44085 (local address 127.9.133.130:33645)
0807 06:15:00.608522 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.608529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.609028 (+ 499us) server_negotiation.cc:407] Connection header received
0807 06:15:00.609265 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.609270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.609337 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.609397 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.609890 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.610597 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.612328 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.612991 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.613101 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.613151 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.614042 (+ 891us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.614159 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":84}
I20260807 06:15:00.614838 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.608140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33645 (local address 127.9.133.131:44085)
0807 06:15:00.608861 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.608932 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.609110 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.609567 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.609571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.609590 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.609727 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.609735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.610746 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.610753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.612190 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.612200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.613644 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.613647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.613746 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.613769 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.613838 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.614674 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":323,"thread_start_us":196,"threads_started":1}
I20260807 06:15:00.614758 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "8584d84f032d4005903758430c8cff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51469ae9114174ad5b714981331150" is_pre_election: true
I20260807 06:15:00.615243 29911 raft_consensus.cc:2454] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8584d84f032d4005903758430c8cff03 in term 0.
I20260807 06:15:00.615495 30410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.608924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42921 (local address 127.9.133.130:47321)
0807 06:15:00.609562 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.609577 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.609671 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.610328 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.610331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.610349 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.610477 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.610483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.611858 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.611861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.613329 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.613342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.614912 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.614916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.615106 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.615135 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.615196 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.615276 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":89,"thread_start_us":74,"threads_started":1}
I20260807 06:15:00.615729 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.609874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:49047 (local address 127.9.133.131:36105)
0807 06:15:00.610299 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.610304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.610317 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:15:00.610347 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.610350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.610390 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.610453 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.610817 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.611582 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.613386 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.614938 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.615097 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.615151 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.615499 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.615594 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":684,"server-negotiator.queue_time_us":341}
I20260807 06:15:00.615896 30108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "8584d84f032d4005903758430c8cff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" is_pre_election: true
I20260807 06:15:00.616075 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.609039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:47321 (local address 127.9.133.129:42921)
0807 06:15:00.610083 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.610092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.610106 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:15:00.610152 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.610155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.610200 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.610573 (+ 373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:00.610724 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.611691 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.614040 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.614738 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.615709 (+ 971us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.615773 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.615840 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.615924 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968,"thread_start_us":141,"threads_started":1}
I20260807 06:15:00.616115 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.609520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:36105 (local address 127.9.133.130:49047)
0807 06:15:00.609939 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.609952 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:00.610039 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:00.610544 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:00.610547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:00.610564 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:00.610685 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.610692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.611728 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.611731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.613246 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:00.613256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.615235 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.615239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:00.615325 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.615350 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:15:00.615880 (+ 530us) client_negotiation.cc:240] Negotiation successful
0807 06:15:00.615960 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":52,"thread_start_us":97,"threads_started":1}
I20260807 06:15:00.616156 30108 raft_consensus.cc:2454] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8584d84f032d4005903758430c8cff03 in term 0.
I20260807 06:15:00.616539 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51469ae9114174ad5b714981331150" is_pre_election: true
I20260807 06:15:00.616699 29911 raft_consensus.cc:2454] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9980c3aa9b8a4ace816a532c0ffa8800 in term 0.
I20260807 06:15:00.616761 30208 leader_election.cc:302] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8584d84f032d4005903758430c8cff03, 9980c3aa9b8a4ace816a532c0ffa8800; no voters:
I20260807 06:15:00.617007 30306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8584d84f032d4005903758430c8cff03" is_pre_election: true
I20260807 06:15:00.617127 30014 leader_election.cc:302] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51469ae9114174ad5b714981331150, 9980c3aa9b8a4ace816a532c0ffa8800; no voters:
I20260807 06:15:00.617221 30306 raft_consensus.cc:2454] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9980c3aa9b8a4ace816a532c0ffa8800 in term 0.
I20260807 06:15:00.617458 30407 raft_consensus.cc:2781] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:15:00.617443 30402 raft_consensus.cc:2781] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:15:00.617560 30407 raft_consensus.cc:482] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:15:00.617560 30402 raft_consensus.cc:482] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:15:00.617624 30407 raft_consensus.cc:3037] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:00.617624 30402 raft_consensus.cc:3037] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:00.619336 30407 raft_consensus.cc:504] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.619330 30402 raft_consensus.cc:504] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.619792 30407 leader_election.cc:288] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [CANDIDATE]: Term 1 election: Requested vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921)
I20260807 06:15:00.619793 30402 leader_election.cc:288] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [CANDIDATE]: Term 1 election: Requested vote from peers 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
I20260807 06:15:00.620205 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "8584d84f032d4005903758430c8cff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51469ae9114174ad5b714981331150"
I20260807 06:15:00.620321 29909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51469ae9114174ad5b714981331150"
I20260807 06:15:00.620386 30306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8584d84f032d4005903758430c8cff03"
I20260807 06:15:00.620398 30108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedd847a851647e8a452c92f4fefa121" candidate_uuid: "8584d84f032d4005903758430c8cff03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
I20260807 06:15:00.620368 29911 raft_consensus.cc:3037] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:00.620553 30306 raft_consensus.cc:2379] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9980c3aa9b8a4ace816a532c0ffa8800 in current term 1: Already voted for candidate 8584d84f032d4005903758430c8cff03 in this term.
I20260807 06:15:00.620571 30108 raft_consensus.cc:2379] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8584d84f032d4005903758430c8cff03 in current term 1: Already voted for candidate 9980c3aa9b8a4ace816a532c0ffa8800 in this term.
I20260807 06:15:00.621176 30016 leader_election.cc:302] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9980c3aa9b8a4ace816a532c0ffa8800; no voters: 4f51469ae9114174ad5b714981331150, 8584d84f032d4005903758430c8cff03
I20260807 06:15:00.621415 30402 raft_consensus.cc:2726] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 06:15:00.622079 29911 raft_consensus.cc:2454] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8584d84f032d4005903758430c8cff03 in term 1.
I20260807 06:15:00.622431 30210 leader_election.cc:302] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f51469ae9114174ad5b714981331150, 8584d84f032d4005903758430c8cff03; no voters: 9980c3aa9b8a4ace816a532c0ffa8800
I20260807 06:15:00.622671 30407 raft_consensus.cc:2781] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:15:00.622973 30407 raft_consensus.cc:686] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 1 LEADER]: Becoming Leader. State: Replica: 8584d84f032d4005903758430c8cff03, State: Running, Role: LEADER
I20260807 06:15:00.623412 30407 consensus_queue.cc:237] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:00.627547 29692 catalog_manager.cc:5495] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8584d84f032d4005903758430c8cff03 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "8584d84f032d4005903758430c8cff03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } health_report { overall_health: HEALTHY } } }
I20260807 06:15:00.651690 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.647087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.9.133.131:36105)
0807 06:15:00.647258 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:00.647262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:00.647278 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:15:00.647386 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:00.647388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:00.647462 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:00.647583 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:15:00.648029 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.648601 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.649708 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:00.649952 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:00.650072 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:00.650489 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:15:00.651217 (+ 728us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:15:00.651338 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:00.651448 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 06:15:00.651534 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:15:00.659772 30108 raft_consensus.cc:1261] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Refusing update from remote peer 8584d84f032d4005903758430c8cff03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:00.659772 29911 raft_consensus.cc:1261] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Refusing update from remote peer 8584d84f032d4005903758430c8cff03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:00.660646 30407 consensus_queue.cc:1035] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:00.660967 30414 consensus_queue.cc:1035] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:00.670401 30421 mvcc.cc:205] Tried to move back new op lower bound from 7315797199491821568 to 7315797199356518400. Current Snapshot: MvccSnapshot[applied={T|T < 7315797199491821568}]
I20260807 06:15:00.681694 29692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:00.681960 29692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:00.685012 29692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Sending DeleteTablet for 3 replicas of tablet fedd847a851647e8a452c92f4fefa121
I20260807 06:15:00.686088 30081 tablet_service.cc:1514] Processing DeleteTablet for tablet fedd847a851647e8a452c92f4fefa121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:00 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:59970
I20260807 06:15:00.686069 29886 tablet_service.cc:1514] Processing DeleteTablet for tablet fedd847a851647e8a452c92f4fefa121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:00 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:51126
I20260807 06:15:00.686662 30277 tablet_service.cc:1514] Processing DeleteTablet for tablet fedd847a851647e8a452c92f4fefa121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:00 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44224
I20260807 06:15:00.686805 30433 tablet_replica.cc:330] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: stopping tablet replica
I20260807 06:15:00.687204 30436 tablet_replica.cc:330] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: stopping tablet replica
I20260807 06:15:00.687316 30433 raft_consensus.cc:2229] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:00.687568 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29785
I20260807 06:15:00.687794 30433 raft_consensus.cc:2258] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:00.693789 30436 raft_consensus.cc:2229] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:00.694470 30436 raft_consensus.cc:2258] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:00.696457 30438 tablet_replica.cc:330] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: stopping tablet replica
I20260807 06:15:00.696918 30438 raft_consensus.cc:2229] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:15:00.697064 30436 ts_tablet_manager.cc:1905] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:00.697502 30438 raft_consensus.cc:2258] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:00.699374 30438 ts_tablet_manager.cc:1905] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:00.701148 30436 ts_tablet_manager.cc:1918] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:00.701370 30436 log.cc:1196] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/fedd847a851647e8a452c92f4fefa121
I20260807 06:15:00.702397 30436 ts_tablet_manager.cc:1939] T fedd847a851647e8a452c92f4fefa121 P 9980c3aa9b8a4ace816a532c0ffa8800: Deleting consensus metadata
I20260807 06:15:00.703068 30438 ts_tablet_manager.cc:1918] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:00.703202 30438 log.cc:1196] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/fedd847a851647e8a452c92f4fefa121
I20260807 06:15:00.703605 30438 ts_tablet_manager.cc:1939] T fedd847a851647e8a452c92f4fefa121 P 8584d84f032d4005903758430c8cff03: Deleting consensus metadata
I20260807 06:15:00.704874 29676 catalog_manager.cc:4841] TS 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645): tablet fedd847a851647e8a452c92f4fefa121 (table test-table [id=6c932937608c4cfd85ab625b786de499]) successfully deleted
I20260807 06:15:00.704921 29680 catalog_manager.cc:4841] TS 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105): tablet fedd847a851647e8a452c92f4fefa121 (table test-table [id=6c932937608c4cfd85ab625b786de499]) successfully deleted
W20260807 06:15:00.705633 29677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:42921 (error 108)
I20260807 06:15:00.706032 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29990
I20260807 06:15:00.715830 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:00.714907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42921 (local address 127.0.0.1:51142)
0807 06:15:00.715462 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:00.715651 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:42921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
W20260807 06:15:00.716137 29677 catalog_manager.cc:4584] TS 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fedd847a851647e8a452c92f4fefa121: Network error: Client connection negotiation failed: client connection to 127.9.133.129:42921: connect: Connection refused (error 111)
I20260807 06:15:00.745357 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 30186
I20260807 06:15:00.758769 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 29656
I20260807 06:15:00.773887 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:32927
--webserver_interface=127.9.133.190
--webserver_port=38847
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:00.920465 30442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:00.920866 30442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:00.920919 30442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:00.925283 30442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:15:00.925356 30442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:00.925377 30442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:00.925395 30442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:15:00.925410 30442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:15:00.931337 30442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:00.932845 30442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:00.934302 30442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:00.952085 30455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:00.952085 30450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:00.952134 30449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:00.952463 30442 server_base.cc:1034] running on GCE node
I20260807 06:15:00.953346 30442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:00.955139 30442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:00.956387 30442 hybrid_clock.cc:648] HybridClock initialized: now 1786083300956372 us; error 43 us; skew 500 ppm
Aug 07 06:15:00 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:00.960462 30442 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:15:00.961764 30442 webserver.cc:460] Webserver started at http://127.9.133.190:38847/ using document root <none> and password file <none>
I20260807 06:15:00.962545 30442 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:00.962677 30442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:00.967900 30442 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:15:00.970957 30464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:00.972774 30442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:15:00.972970 30442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "161e481d0e184eb8bf6bbfc6d27cf30c"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "a3338763e4cc59791df04b499922a4d6"
server_key_iv: "25e2e46372153ef019f1f0d147154c6e"
server_key_version: "encryptionkey@0"
I20260807 06:15:00.973770 30442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:01.032516 30442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:01.037387 30442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:01.037680 30442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:01.047132 30442 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:32927
I20260807 06:15:01.047473 30534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:32927 every 8 connection(s)
I20260807 06:15:01.048656 30442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:15:01.054629 30535 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:15:01.055455 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 30442
I20260807 06:15:01.056093 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:42921
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:32927
--webserver_port=36001
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:01.057817 30535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap starting.
I20260807 06:15:01.081938 30535 log.cc:824] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:01.093330 30535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:15:01.093889 30535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap complete.
I20260807 06:15:01.099387 30535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:01.100117 30535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161e481d0e184eb8bf6bbfc6d27cf30c, State: Initialized, Role: FOLLOWER
I20260807 06:15:01.100962 30535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:01.101153 30535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:15:01.101276 30535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:15:01.101413 30535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:15:01.104485 30535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:01.105105 30535 leader_election.cc:302] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161e481d0e184eb8bf6bbfc6d27cf30c; no voters:
I20260807 06:15:01.105623 30535 leader_election.cc:288] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:15:01.105743 30541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:15:01.106035 30541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 LEADER]: Becoming Leader. State: Replica: 161e481d0e184eb8bf6bbfc6d27cf30c, State: Running, Role: LEADER
I20260807 06:15:01.106431 30541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:01.107334 30535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:15:01.108218 30542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } } }
I20260807 06:15:01.108364 30542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:01.109076 30543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161e481d0e184eb8bf6bbfc6d27cf30c. Latest consensus state: current_term: 2 leader_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } } }
I20260807 06:15:01.109180 30543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:01.109503 30548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:15:01.116047 30548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:15:01.116781 30548 catalog_manager.cc:1259] Loaded cluster ID: bb674bb300694cc0a60d67fb2223968e
I20260807 06:15:01.117018 30548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:15:01.118633 30548 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 06:15:01.123088 30548 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 06:15:01.130383 30548 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:15:01.131461 30548 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Loaded TSK: 0
I20260807 06:15:01.132979 30548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:15:01.208751 30538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:01.209223 30538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:01.209324 30538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:01.213888 30538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:01.214033 30538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:01.214141 30538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:15:01.220571 30538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:42921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=36001
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:01.222007 30538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:01.223436 30538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:01.235244 30572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:01.235231 30571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:01.235563 30538 server_base.cc:1034] running on GCE node
W20260807 06:15:01.235240 30575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:01.236343 30538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:01.237232 30538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:01.238430 30538 hybrid_clock.cc:648] HybridClock initialized: now 1786083301238414 us; error 45 us; skew 500 ppm
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:01.241863 30538 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:15:01.243132 30538 webserver.cc:460] Webserver started at http://127.9.133.129:36001/ using document root <none> and password file <none>
I20260807 06:15:01.243877 30538 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:01.243991 30538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:01.248929 30538 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 06:15:01.251753 30585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.253400 30538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:01.253585 30538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "4f51469ae9114174ad5b714981331150"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "b6c12924bd1adfefa263b762bf845d5c"
server_key_iv: "d29a54d4b315c1e44f275d6701c68b46"
server_key_version: "encryptionkey@0"
I20260807 06:15:01.254302 30538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:01.273355 30538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:01.277776 30538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:01.278061 30538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:01.278923 30538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:01.280366 30596 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:15:01.285782 30538 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:15:01.285897 30538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 06:15:01.285997 30538 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:15:01.288208 30538 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:15:01.288287 30538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 06:15:01.288719 30596 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Bootstrap starting.
I20260807 06:15:01.301191 30538 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:42921
I20260807 06:15:01.301268 30746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:42921 every 8 connection(s)
I20260807 06:15:01.302449 30538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:15:01.307245 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 30538
I20260807 06:15:01.307781 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:33645
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:32927
--webserver_port=35999
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:01.312963 30596 log.cc:824] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:01.323285 30596 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:15:01.323817 30596 tablet_bootstrap.cc:492] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Bootstrap complete.
I20260807 06:15:01.325512 30596 ts_tablet_manager.cc:1397] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Time spent bootstrapping tablet: real 0.037s user 0.030s sys 0.004s
I20260807 06:15:01.325744 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.305219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:35085)
0807 06:15:01.306033 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.306128 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.307649 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.309612 (+ 1963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.309626 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.310279 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:01.310914 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.310934 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.312478 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.312481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.313573 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.313588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.314862 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.314866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.315146 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.316023 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:01.316047 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:01.318529 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:01.321463 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.321473 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.321492 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.321920 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.322384 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.322389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.322390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.322603 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.323524 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:01.323534 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:01.323890 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.324150 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.324578 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":218,"threads_started":1}
I20260807 06:15:01.325757 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.305225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:35085 (local address 127.9.133.190:32927)
0807 06:15:01.305742 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.305767 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.306313 (+ 546us) server_negotiation.cc:407] Connection header received
0807 06:15:01.308467 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.308485 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.309001 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.309571 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:01.311167 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.312279 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.313726 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.314614 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.314835 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.318792 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:01.318800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:01.318817 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:01.318854 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:01.321317 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.322080 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.322087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.322092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.322260 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.322738 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.322742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.322743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.323299 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:01.323450 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.324323 (+ 873us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.324680 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":154,"threads_started":1}
I20260807 06:15:01.328740 30596 raft_consensus.cc:348] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:01.329317 30596 raft_consensus.cc:729] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51469ae9114174ad5b714981331150, State: Initialized, Role: FOLLOWER
I20260807 06:15:01.329711 30747 heartbeater.cc:347] Connected to a master server at 127.9.133.190:32927
I20260807 06:15:01.329900 30596 consensus_queue.cc:260] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } }
I20260807 06:15:01.329986 30747 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:01.330694 30747 heartbeater.cc:511] Master 127.9.133.190:32927 requested a full tablet report, sending...
I20260807 06:15:01.331140 30596 ts_tablet_manager.cc:1428] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260807 06:15:01.334331 30487 ts_manager.cc:194] Registered new tserver with Master: 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921)
I20260807 06:15:01.339288 30487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:35085
I20260807 06:15:01.342447 30747 heartbeater.cc:503] Master 127.9.133.190:32927 was elected leader, sending a full tablet report...
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.129@KRBTEST.COM
I20260807 06:15:01.350605 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.338600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42921 (local address 127.0.0.1:51150)
0807 06:15:01.339642 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.339684 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.339879 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.340536 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.340542 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.340965 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:01.341090 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.341097 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.342097 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.342101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.342884 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.342895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.343713 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.343716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.343825 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.344598 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:01.344613 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:01.346430 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:01.348772 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.348780 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.348785 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.349037 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.349366 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.349369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.349370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.349456 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.350019 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:01.350024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:01.350139 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.350260 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.350439 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":181,"threads_started":1}
I20260807 06:15:01.350764 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.338695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.9.133.129:42921)
0807 06:15:01.339252 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.339266 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.339804 (+ 538us) server_negotiation.cc:407] Connection header received
0807 06:15:01.340089 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.340094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.340359 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.340521 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:01.341221 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.341967 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.343040 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.343569 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.343688 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.346585 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:01.346595 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:01.346611 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:01.346639 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:01.348649 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.349155 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.349161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.349163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.349267 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.349545 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.349548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.349550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.349884 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:01.350026 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.350378 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.350585 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":148,"threads_started":1}
I20260807 06:15:01.351920 30655 tablet_service.cc:1514] Processing DeleteTablet for tablet fedd847a851647e8a452c92f4fefa121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:00 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:51150
I20260807 06:15:01.353015 30760 tablet_replica.cc:330] stopping tablet replica
I20260807 06:15:01.353431 30760 raft_consensus.cc:2229] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:01.353761 30760 raft_consensus.cc:2258] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:01.355365 30760 ts_tablet_manager.cc:1905] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:01.361143 30760 ts_tablet_manager.cc:1918] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:01.361277 30760 log.cc:1196] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/fedd847a851647e8a452c92f4fefa121
I20260807 06:15:01.361622 30760 ts_tablet_manager.cc:1939] T fedd847a851647e8a452c92f4fefa121 P 4f51469ae9114174ad5b714981331150: Deleting consensus metadata
I20260807 06:15:01.364231 30466 catalog_manager.cc:4841] TS 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921): tablet fedd847a851647e8a452c92f4fefa121 (table test-table [id=6c932937608c4cfd85ab625b786de499]) successfully deleted
W20260807 06:15:01.461338 30753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:01.461773 30753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:01.461866 30753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:01.466343 30753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:01.466480 30753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:01.466586 30753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:15:01.473220 30753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:33645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=35999
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:01.474844 30753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:01.476346 30753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:01.489194 30776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:01.489377 30753 server_base.cc:1034] running on GCE node
W20260807 06:15:01.489215 30770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:01.489215 30774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:01.490042 30753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:01.491070 30753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:01.492312 30753 hybrid_clock.cc:648] HybridClock initialized: now 1786083301492269 us; error 57 us; skew 500 ppm
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:01.496126 30753 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:15:01.497430 30753 webserver.cc:460] Webserver started at http://127.9.133.130:35999/ using document root <none> and password file <none>
I20260807 06:15:01.498405 30753 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:01.498548 30753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:01.504189 30753 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:15:01.506944 30786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.508828 30753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:01.508972 30753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "d953a446c00e7483336812b9a8b7e3da"
server_key_iv: "5d3de85efb17a33ddfa494848bd06493"
server_key_version: "encryptionkey@0"
I20260807 06:15:01.509720 30753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:01.524979 30753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:01.529191 30753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:01.529419 30753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:01.530218 30753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:01.531365 30753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:01.531428 30753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.531477 30753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:01.531493 30753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.543076 30753 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:33645
I20260807 06:15:01.543134 30972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:33645 every 8 connection(s)
I20260807 06:15:01.544477 30753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:15:01.549762 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 30753
I20260807 06:15:01.550391 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:36105
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:32927
--webserver_port=38079
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:01.561367 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.546865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:53235 (local address 127.9.133.190:32927)
0807 06:15:01.547129 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.547134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.548450 (+ 1316us) server_negotiation.cc:407] Connection header received
0807 06:15:01.549572 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.549576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.549640 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.549724 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:01.551695 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.552295 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.553780 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.554051 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.554175 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.557115 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:01.557119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:01.557121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:01.557152 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:01.558974 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.559582 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.559586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.559588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.559681 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.560060 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.560063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.560065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.560285 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:01.560400 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.561001 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.561163 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 06:15:01.562536 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.547435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:53235)
0807 06:15:01.548271 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.548320 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.549290 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.550001 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.550011 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.550630 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:01.551496 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.551523 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.552437 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.552443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.553625 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.553635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.554250 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.554255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.554442 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.555198 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:01.555217 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:01.556901 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:01.559145 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.559152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.559172 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.559464 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.559777 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.559781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.559783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.559944 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.560398 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:01.560406 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:01.560718 (+ 312us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.560933 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.561350 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":137,"threads_started":1}
I20260807 06:15:01.564081 30973 heartbeater.cc:347] Connected to a master server at 127.9.133.190:32927
I20260807 06:15:01.564402 30973 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:01.565052 30973 heartbeater.cc:511] Master 127.9.133.190:32927 requested a full tablet report, sending...
I20260807 06:15:01.566370 30487 ts_manager.cc:194] Registered new tserver with Master: 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645)
I20260807 06:15:01.567293 30487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:53235
W20260807 06:15:01.688830 30976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:01.689224 30976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:01.689273 30976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:01.693657 30976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:01.693764 30976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:01.693835 30976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:15:01.700290 30976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:36105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=38079
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:01.701856 30976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:01.703613 30976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:01.716602 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:01.716656 30988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:01.716656 30982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:01.717144 30976 server_base.cc:1034] running on GCE node
I20260807 06:15:01.717724 30976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:01.718632 30976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:01.719870 30976 hybrid_clock.cc:648] HybridClock initialized: now 1786083301719854 us; error 64 us; skew 500 ppm
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:01.723731 30976 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:15:01.724960 30976 webserver.cc:460] Webserver started at http://127.9.133.131:38079/ using document root <none> and password file <none>
I20260807 06:15:01.725723 30976 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:01.725850 30976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:01.730846 30976 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:15:01.733621 30999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.735448 30976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:01.735634 30976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "8584d84f032d4005903758430c8cff03"
format_stamp: "Formatted at 2026-08-07 06:15:00 on dist-test-slave-h3ft"
server_key: "4c4b034b3bb8a81c0dd6aec9e0ba1ca9"
server_key_iv: "6f98fa5f082ae2a239ea62b188c55d93"
server_key_version: "encryptionkey@0"
I20260807 06:15:01.736418 30976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:01.757942 30976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:01.762290 30976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:01.762547 30976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:01.763424 30976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:01.764478 30976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:01.764559 30976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.764652 30976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:01.764703 30976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:01.776257 30976 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:36105
I20260807 06:15:01.776278 31156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:36105 every 8 connection(s)
I20260807 06:15:01.777758 30976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:15:01.781404 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 30976
Aug 07 06:15:01 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:01.792281 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.782370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.9.133.190:32927)
0807 06:15:01.782692 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.782696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.782806 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:15:01.782956 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.782961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.783066 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.783181 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:01.784102 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.784963 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.786002 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.786262 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.786388 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.788185 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:01.788189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:01.788192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:01.788229 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:01.789912 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.790400 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.790404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.790406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.790507 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.790816 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.790820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.790822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.791183 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:01.791465 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.791871 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.792043 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
I20260807 06:15:01.794988 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.780273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:53673 (local address 127.9.133.190:32927)
0807 06:15:01.780465 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.780470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.781980 (+ 1510us) server_negotiation.cc:407] Connection header received
0807 06:15:01.783433 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.783436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.783488 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.783552 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:01.785407 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.785933 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.787332 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.787579 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.787693 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.790667 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:01.790670 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:01.790672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:01.790694 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:01.792282 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.793098 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.793104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.793105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.793194 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:01.793622 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:01.793625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:01.793644 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:01.793883 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:01.793989 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.794692 (+ 703us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.794845 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:15:01.796295 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.780723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:53673)
0807 06:15:01.781729 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.781796 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.783158 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.783797 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.783806 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.784385 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:01.785252 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.785272 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.786082 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.786091 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.787187 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.787198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.787773 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.787780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.787989 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.788662 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:01.788682 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:01.790465 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:01.792535 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.792543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.792564 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.793003 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.793297 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:01.793300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:01.793302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:01.793516 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:01.794001 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:01.794009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:01.794361 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.794631 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.794991 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":149,"threads_started":1}
I20260807 06:15:01.797827 31157 heartbeater.cc:347] Connected to a master server at 127.9.133.190:32927
I20260807 06:15:01.798205 31157 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:01.798879 31157 heartbeater.cc:511] Master 127.9.133.190:32927 requested a full tablet report, sending...
I20260807 06:15:01.800249 30487 ts_manager.cc:194] Registered new tserver with Master: 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
I20260807 06:15:01.801091 30487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:53673
I20260807 06:15:01.807263 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:15:01.813055 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.808189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.9.133.190:32927)
0807 06:15:01.808395 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.808401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.808720 (+ 319us) server_negotiation.cc:407] Connection header received
0807 06:15:01.808955 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.808958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.809135 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.809273 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:15:01.809650 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.810231 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.811262 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.811502 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.811624 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.812017 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:15:01.812563 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:15:01.812748 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.812784 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.812858 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:15:01.813978 30487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:15:01.818359 30487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:15:01.830266 30655 tablet_service.cc:1467] Processing CreateTablet for tablet 0b7bc99c4938414fac85b0ff4bb31600 (DEFAULT_TABLE table=test-table [id=84419807f7ff47ac9a3cdf4801d82e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:01.831297 30655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b7bc99c4938414fac85b0ff4bb31600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:01.835410 31172 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Bootstrap starting.
I20260807 06:15:01.835995 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.829114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:36105 (local address 127.0.0.1:44240)
0807 06:15:01.829483 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.829505 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.829605 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.830830 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.830836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.830857 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:01.831036 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.831044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.833187 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.833191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.834283 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.834295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.835434 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.835437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.835552 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.835638 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.835695 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.835793 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":5,"thread_start_us":177,"threads_started":1}
I20260807 06:15:01.836535 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.828811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33645 (local address 127.0.0.1:59986)
0807 06:15:01.829041 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.829078 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.829211 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.830885 (+ 1674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.830889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.830910 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:01.831066 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.831073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.832734 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.832740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.834068 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.834080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.835395 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.835402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.836184 (+ 782us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.836215 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.836306 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.836410 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":640}
I20260807 06:15:01.836673 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.829455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.9.133.130:33645)
0807 06:15:01.829930 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.829946 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.829976 (+ 30us) server_negotiation.cc:407] Connection header received
0807 06:15:01.830083 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.830275 (+ 192us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.830694 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.831381 (+ 687us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:01.831588 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.832563 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.834221 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.835228 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.835715 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.835918 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.836358 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.836490 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260807 06:15:01.836766 31172 tablet_bootstrap.cc:654] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:01.837387 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.829550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.9.133.131:36105)
0807 06:15:01.830206 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.830220 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.830246 (+ 26us) server_negotiation.cc:407] Connection header received
0807 06:15:01.830342 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.830346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.830614 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.831229 (+ 615us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:01.831429 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.833016 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.834438 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.835289 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.836858 (+ 1569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.837047 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.837103 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.837217 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":194,"threads_started":1}
I20260807 06:15:01.838796 31172 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: No bootstrap required, opened a new log
I20260807 06:15:01.838923 31172 ts_tablet_manager.cc:1397] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 06:15:01.839489 31172 raft_consensus.cc:348] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.839651 31172 raft_consensus.cc:374] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:01.839694 31172 raft_consensus.cc:729] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51469ae9114174ad5b714981331150, State: Initialized, Role: FOLLOWER
I20260807 06:15:01.839846 31172 consensus_queue.cc:260] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.840255 31172 ts_tablet_manager.cc:1428] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 06:15:01.840785 31068 tablet_service.cc:1467] Processing CreateTablet for tablet 0b7bc99c4938414fac85b0ff4bb31600 (DEFAULT_TABLE table=test-table [id=84419807f7ff47ac9a3cdf4801d82e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:01.840792 30856 tablet_service.cc:1467] Processing CreateTablet for tablet 0b7bc99c4938414fac85b0ff4bb31600 (DEFAULT_TABLE table=test-table [id=84419807f7ff47ac9a3cdf4801d82e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:01.842243 30856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b7bc99c4938414fac85b0ff4bb31600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:01.842262 31068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b7bc99c4938414fac85b0ff4bb31600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:01.851125 31178 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Bootstrap starting.
I20260807 06:15:01.851125 31175 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Bootstrap starting.
I20260807 06:15:01.854471 31175 tablet_bootstrap.cc:654] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:01.854470 31178 tablet_bootstrap.cc:654] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:01.855531 31178 log.cc:824] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:01.855530 31175 log.cc:824] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:01.858016 31175 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: No bootstrap required, opened a new log
I20260807 06:15:01.858305 31175 ts_tablet_manager.cc:1397] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260807 06:15:01.859764 31178 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: No bootstrap required, opened a new log
I20260807 06:15:01.860028 31178 ts_tablet_manager.cc:1397] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 06:15:01.862679 31175 raft_consensus.cc:348] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.863118 31175 raft_consensus.cc:374] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:01.863199 31175 raft_consensus.cc:729] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9980c3aa9b8a4ace816a532c0ffa8800, State: Initialized, Role: FOLLOWER
I20260807 06:15:01.863891 31175 consensus_queue.cc:260] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.864574 31178 raft_consensus.cc:348] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.864909 30973 heartbeater.cc:503] Master 127.9.133.190:32927 was elected leader, sending a full tablet report...
I20260807 06:15:01.864971 31178 raft_consensus.cc:374] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:01.865051 31178 raft_consensus.cc:729] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8584d84f032d4005903758430c8cff03, State: Initialized, Role: FOLLOWER
I20260807 06:15:01.865345 31175 ts_tablet_manager.cc:1428] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260807 06:15:01.865617 31178 consensus_queue.cc:260] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.866533 31157 heartbeater.cc:503] Master 127.9.133.190:32927 was elected leader, sending a full tablet report...
I20260807 06:15:01.866803 31178 ts_tablet_manager.cc:1428] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 06:15:01.963107 31174 raft_consensus.cc:482] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:01.963457 31174 raft_consensus.cc:504] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.965123 31174 leader_election.cc:288] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
I20260807 06:15:01.971616 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.965496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:36105 (local address 127.9.133.129:38979)
0807 06:15:01.966054 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.966077 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.966358 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.966790 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.966793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.966819 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:01.966948 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.966955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.967976 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.967982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.969703 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.969722 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.970908 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.970911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.971095 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.971119 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.971220 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.971365 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":37,"spinlock_wait_cycles":19328,"thread_start_us":160,"threads_started":1}
I20260807 06:15:01.971604 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.965699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:38979 (local address 127.9.133.131:36105)
0807 06:15:01.965999 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.966007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.966190 (+ 183us) server_negotiation.cc:407] Connection header received
0807 06:15:01.966553 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.966559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.966642 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.966755 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:01.967142 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.967828 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.969879 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.970591 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.970742 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.970820 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.971265 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.971388 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260807 06:15:01.971675 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.965496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33645 (local address 127.9.133.129:55737)
0807 06:15:01.966158 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:01.966173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:01.966358 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:01.967344 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:01.967350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:01.967380 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:01.967572 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.967583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.968547 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.968554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.969572 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:01.969591 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.970755 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.970761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:01.970927 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.971094 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 06:15:01.971198 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 06:15:01.971375 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":375,"spinlock_wait_cycles":35456,"thread_start_us":103,"threads_started":1}
I20260807 06:15:01.972851 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:01.965764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:55737 (local address 127.9.133.130:33645)
0807 06:15:01.966148 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:01.966154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:01.966273 (+ 119us) server_negotiation.cc:407] Connection header received
0807 06:15:01.966978 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:01.966983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:01.967092 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:01.967180 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:01.967749 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.968389 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.969762 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:01.970560 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:01.970742 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:01.970820 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:01.972009 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0807 06:15:01.972146 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260807 06:15:01.973470 31094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b7bc99c4938414fac85b0ff4bb31600" candidate_uuid: "4f51469ae9114174ad5b714981331150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8584d84f032d4005903758430c8cff03" is_pre_election: true
I20260807 06:15:01.973701 30882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b7bc99c4938414fac85b0ff4bb31600" candidate_uuid: "4f51469ae9114174ad5b714981331150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" is_pre_election: true
I20260807 06:15:01.974069 31094 raft_consensus.cc:2454] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51469ae9114174ad5b714981331150 in term 0.
I20260807 06:15:01.974125 30882 raft_consensus.cc:2454] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51469ae9114174ad5b714981331150 in term 0.
I20260807 06:15:01.974849 30586 leader_election.cc:302] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51469ae9114174ad5b714981331150, 9980c3aa9b8a4ace816a532c0ffa8800; no voters:
I20260807 06:15:01.975387 31174 raft_consensus.cc:2781] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:15:01.975553 31174 raft_consensus.cc:482] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:15:01.975616 31174 raft_consensus.cc:3037] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:01.977053 31174 raft_consensus.cc:504] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.977675 31174 leader_election.cc:288] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 1 election: Requested vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
I20260807 06:15:01.978286 30882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b7bc99c4938414fac85b0ff4bb31600" candidate_uuid: "4f51469ae9114174ad5b714981331150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
I20260807 06:15:01.978510 30882 raft_consensus.cc:3037] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:01.978502 31094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b7bc99c4938414fac85b0ff4bb31600" candidate_uuid: "4f51469ae9114174ad5b714981331150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8584d84f032d4005903758430c8cff03"
I20260807 06:15:01.978721 31094 raft_consensus.cc:3037] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:01.980283 31094 raft_consensus.cc:2454] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51469ae9114174ad5b714981331150 in term 1.
I20260807 06:15:01.980808 30882 raft_consensus.cc:2454] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51469ae9114174ad5b714981331150 in term 1.
I20260807 06:15:01.981056 30590 leader_election.cc:302] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f51469ae9114174ad5b714981331150, 8584d84f032d4005903758430c8cff03; no voters:
I20260807 06:15:01.981371 31174 raft_consensus.cc:2781] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:15:01.981684 31174 raft_consensus.cc:686] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 LEADER]: Becoming Leader. State: Replica: 4f51469ae9114174ad5b714981331150, State: Running, Role: LEADER
I20260807 06:15:01.982121 31174 consensus_queue.cc:237] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:01.987516 30487 catalog_manager.cc:5495] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f51469ae9114174ad5b714981331150 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "4f51469ae9114174ad5b714981331150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } health_report { overall_health: HEALTHY } } }
W20260807 06:15:02.029124 31158 tablet.cc:2369] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:15:02.041478 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.035332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.9.133.129:42921)
0807 06:15:02.035788 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:02.035801 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:02.035824 (+ 23us) server_negotiation.cc:407] Connection header received
0807 06:15:02.035985 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:02.035992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:02.036099 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:02.036293 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:15:02.036611 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:02.037638 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:02.038602 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:02.039149 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:02.039289 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:02.039567 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:15:02.040502 (+ 935us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:15:02.040650 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:02.040703 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 06:15:02.040801 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":167,"threads_started":1}
W20260807 06:15:02.046113 30974 tablet.cc:2369] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:15:02.048781 30882 raft_consensus.cc:1261] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Refusing update from remote peer 4f51469ae9114174ad5b714981331150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:02.048782 31094 raft_consensus.cc:1261] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Refusing update from remote peer 4f51469ae9114174ad5b714981331150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:02.049906 31187 consensus_queue.cc:1035] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:02.050283 31174 consensus_queue.cc:1035] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:15:02.054409 30749 tablet.cc:2369] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:15:02.065689 31196 mvcc.cc:205] Tried to move back new op lower bound from 7315797205177581568 to 7315797204923699200. Current Snapshot: MvccSnapshot[applied={T|T < 7315797205177581568}]
I20260807 06:15:02.073660 31198 mvcc.cc:205] Tried to move back new op lower bound from 7315797205177581568 to 7315797204923699200. Current Snapshot: MvccSnapshot[applied={T|T < 7315797205177581568}]
I20260807 06:15:02.079495 31199 mvcc.cc:205] Tried to move back new op lower bound from 7315797205177581568 to 7315797204923699200. Current Snapshot: MvccSnapshot[applied={T|T < 7315797205177581568 or (T in {7315797205177581568})}]
I20260807 06:15:02.083979 30486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:02.084267 30486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:02.087913 30486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Sending DeleteTablet for 3 replicas of tablet 0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:02.089143 30856 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b7bc99c4938414fac85b0ff4bb31600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:02 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:59986
I20260807 06:15:02.089960 31211 tablet_replica.cc:330] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: stopping tablet replica
I20260807 06:15:02.090044 31068 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b7bc99c4938414fac85b0ff4bb31600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:02 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44240
I20260807 06:15:02.090782 31211 raft_consensus.cc:2229] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:02.091450 31211 raft_consensus.cc:2258] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:02.091464 30655 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b7bc99c4938414fac85b0ff4bb31600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:02 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:51150
I20260807 06:15:02.093348 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 30538
I20260807 06:15:02.094018 31211 ts_tablet_manager.cc:1905] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:02.099788 31211 ts_tablet_manager.cc:1918] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:02.100190 31211 log.cc:1196] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:02.100742 31211 ts_tablet_manager.cc:1939] T 0b7bc99c4938414fac85b0ff4bb31600 P 9980c3aa9b8a4ace816a532c0ffa8800: Deleting consensus metadata
I20260807 06:15:02.103068 30465 catalog_manager.cc:4841] TS 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645): tablet 0b7bc99c4938414fac85b0ff4bb31600 (table test-table [id=84419807f7ff47ac9a3cdf4801d82e33]) successfully deleted
I20260807 06:15:02.107568 31215 tablet_replica.cc:330] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: stopping tablet replica
I20260807 06:15:02.108237 31215 raft_consensus.cc:2229] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:02.108882 31215 raft_consensus.cc:2258] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:02.111220 31215 ts_tablet_manager.cc:1905] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:02.115671 31215 ts_tablet_manager.cc:1918] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:02.116189 31215 log.cc:1196] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:02.116719 31215 ts_tablet_manager.cc:1939] T 0b7bc99c4938414fac85b0ff4bb31600 P 8584d84f032d4005903758430c8cff03: Deleting consensus metadata
W20260807 06:15:02.117934 30466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:42921 (error 108)
I20260807 06:15:02.118503 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 30753
I20260807 06:15:02.122630 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.122091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42921 (local address 127.0.0.1:51168)
0807 06:15:02.122339 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.122470 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:42921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 06:15:02.122663 30469 catalog_manager.cc:4841] TS 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105): tablet 0b7bc99c4938414fac85b0ff4bb31600 (table test-table [id=84419807f7ff47ac9a3cdf4801d82e33]) successfully deleted
W20260807 06:15:02.122879 30466 catalog_manager.cc:4584] TS 4f51469ae9114174ad5b714981331150 (127.9.133.129:42921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b7bc99c4938414fac85b0ff4bb31600: Network error: Client connection negotiation failed: client connection to 127.9.133.129:42921: connect: Connection refused (error 111)
I20260807 06:15:02.135294 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 30976
I20260807 06:15:02.156714 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 30442
I20260807 06:15:02.178514 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:32927
--webserver_interface=127.9.133.190
--webserver_port=38847
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:02.336074 31220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:02.336531 31220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:02.336585 31220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:02.341535 31220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:15:02.341674 31220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:02.341697 31220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:02.341718 31220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:15:02.341735 31220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:15:02.348641 31220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:02.350334 31220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:02.351837 31220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:02.364704 31233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:02.364704 31226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:02.365020 31228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:02.365996 31220 server_base.cc:1034] running on GCE node
I20260807 06:15:02.366945 31220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:02.368769 31220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:02.370026 31220 hybrid_clock.cc:648] HybridClock initialized: now 1786083302369983 us; error 71 us; skew 500 ppm
Aug 07 06:15:02 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:02.374807 31220 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:15:02.376569 31220 webserver.cc:460] Webserver started at http://127.9.133.190:38847/ using document root <none> and password file <none>
I20260807 06:15:02.377494 31220 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:02.377596 31220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:02.383675 31220 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:15:02.387476 31243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:02.389753 31220 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:15:02.389986 31220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "161e481d0e184eb8bf6bbfc6d27cf30c"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "a3338763e4cc59791df04b499922a4d6"
server_key_iv: "25e2e46372153ef019f1f0d147154c6e"
server_key_version: "encryptionkey@0"
I20260807 06:15:02.390887 31220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:02.412492 31220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:02.417261 31220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:02.417546 31220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:02.427554 31220 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:32927
I20260807 06:15:02.427953 31329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:32927 every 8 connection(s)
I20260807 06:15:02.429111 31220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:15:02.430567 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 31220
I20260807 06:15:02.431217 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:42921
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:32927
--webserver_port=36001
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:02.435755 31330 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:15:02.438274 31330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap starting.
I20260807 06:15:02.462061 31330 log.cc:824] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:02.475796 31330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:15:02.476358 31330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c: Bootstrap complete.
I20260807 06:15:02.481503 31330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:02.482087 31330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161e481d0e184eb8bf6bbfc6d27cf30c, State: Initialized, Role: FOLLOWER
I20260807 06:15:02.482797 31330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:02.482972 31330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:15:02.483116 31330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:15:02.483263 31330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:15:02.485823 31330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:02.486269 31330 leader_election.cc:302] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161e481d0e184eb8bf6bbfc6d27cf30c; no voters:
I20260807 06:15:02.486721 31330 leader_election.cc:288] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 06:15:02.486845 31334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:15:02.487134 31334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [term 3 LEADER]: Becoming Leader. State: Replica: 161e481d0e184eb8bf6bbfc6d27cf30c, State: Running, Role: LEADER
I20260807 06:15:02.487529 31334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } }
I20260807 06:15:02.488502 31330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:15:02.489317 31335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } } }
I20260807 06:15:02.489488 31335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:02.489877 31336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161e481d0e184eb8bf6bbfc6d27cf30c. Latest consensus state: current_term: 3 leader_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161e481d0e184eb8bf6bbfc6d27cf30c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 32927 } } }
I20260807 06:15:02.490039 31336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161e481d0e184eb8bf6bbfc6d27cf30c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:02.490763 31341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:15:02.497617 31341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:15:02.498466 31341 catalog_manager.cc:1259] Loaded cluster ID: bb674bb300694cc0a60d67fb2223968e
I20260807 06:15:02.498546 31341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 06:15:02.499933 31341 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9e5e30d9ad google::LogMessage::Fail() at ??:0
@ 0x7f9e5e31088a google::LogMessage::SendToLog() at ??:0
@ 0x7f9e5e30d4de google::LogMessage::Flush() at ??:0
@ 0x7f9e5e30ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9e68c4cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9e68c4dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9e68c4a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9e68c8e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9e6055f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9e6055fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9e605613f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9e60550512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9e61b946db start_thread at ??:0
@ 0x7f9e5ce0b61f clone at ??:0
W20260807 06:15:02.614727 31332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:02.615211 31332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:02.615299 31332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:02.620213 31332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:02.620316 31332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:02.620390 31332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:15:02.626993 31332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:42921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=36001
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:02.628659 31332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:02.630100 31332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:02.643518 31385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:02.643564 31384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:02.643595 31387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:02.644344 31332 server_base.cc:1034] running on GCE node
I20260807 06:15:02.645011 31332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:02.646013 31332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:02.647217 31332 hybrid_clock.cc:648] HybridClock initialized: now 1786083302647200 us; error 45 us; skew 500 ppm
Aug 07 06:15:02 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:02.656543 31332 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:15:02.658221 31332 webserver.cc:460] Webserver started at http://127.9.133.129:36001/ using document root <none> and password file <none>
I20260807 06:15:02.659168 31332 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:02.659304 31332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:02.665309 31332 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:15:02.668416 31401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:02.670214 31332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:15:02.670393 31332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "4f51469ae9114174ad5b714981331150"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "b6c12924bd1adfefa263b762bf845d5c"
server_key_iv: "d29a54d4b315c1e44f275d6701c68b46"
server_key_version: "encryptionkey@0"
I20260807 06:15:02.671211 31332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:02.688854 31332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:02.693965 31332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:02.694255 31332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:02.695217 31332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:02.696959 31412 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:15:02.705094 31332 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:15:02.705276 31332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.001s
I20260807 06:15:02.705391 31332 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:15:02.708410 31332 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:15:02.708541 31332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.001s
I20260807 06:15:02.708968 31412 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Bootstrap starting.
I20260807 06:15:02.722434 31332 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:42921
I20260807 06:15:02.722769 31570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:42921 every 8 connection(s)
I20260807 06:15:02.723996 31332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:15:02.733831 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 31332
I20260807 06:15:02.734584 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:33645
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:32927
--webserver_port=35999
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:02.736274 31412 log.cc:824] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:02.748755 31412 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:15:02.749359 31412 tablet_bootstrap.cc:492] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Bootstrap complete.
I20260807 06:15:02.750911 31412 ts_tablet_manager.cc:1397] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Time spent bootstrapping tablet: real 0.042s user 0.021s sys 0.016s
I20260807 06:15:02.755175 31412 raft_consensus.cc:348] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:02.755779 31412 raft_consensus.cc:729] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51469ae9114174ad5b714981331150, State: Initialized, Role: FOLLOWER
I20260807 06:15:02.756479 31412 consensus_queue.cc:260] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:02.757711 31412 ts_tablet_manager.cc:1428] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 06:15:02.764111 31578 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:15:02.727231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:56545)
0807 06:15:02.728026 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.728108 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:02.729281 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:02.762764 (+ 33483us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":240,"threads_started":1}
W20260807 06:15:02.764849 31571 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:32927 (0 consecutive failures): Network error: Failed to ping master at 127.9.133.190:32927: Client connection negotiation failed: client connection to 127.9.133.190:32927: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:15:02.765769 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.765452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:45541)
0807 06:15:02.765604 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.765694 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 06:15:02.766455 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.766241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:33847)
0807 06:15:02.766351 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.766405 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 06:15:02.766634 31571 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:15:02.899323 31579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:02.899791 31579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:02.899890 31579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:02.904503 31579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:02.904668 31579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:02.904796 31579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:15:02.911182 31579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:33645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=35999
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:02.912878 31579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:02.914623 31579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:02.927974 31592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:02.927910 31591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:02.927910 31594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:02.928444 31579 server_base.cc:1034] running on GCE node
I20260807 06:15:02.928956 31579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:02.929854 31579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:02.931102 31579 hybrid_clock.cc:648] HybridClock initialized: now 1786083302931074 us; error 33 us; skew 500 ppm
Aug 07 06:15:02 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:02.934944 31579 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:15:02.936409 31579 webserver.cc:460] Webserver started at http://127.9.133.130:35999/ using document root <none> and password file <none>
I20260807 06:15:02.937258 31579 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:02.937379 31579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:02.942875 31579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 06:15:02.946024 31606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:02.947889 31579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:02.948081 31579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "9980c3aa9b8a4ace816a532c0ffa8800"
format_stamp: "Formatted at 2026-08-07 06:14:59 on dist-test-slave-h3ft"
server_key: "d953a446c00e7483336812b9a8b7e3da"
server_key_iv: "5d3de85efb17a33ddfa494848bd06493"
server_key_version: "encryptionkey@0"
I20260807 06:15:02.948866 31579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:02.964335 31579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:02.969172 31579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:02.969453 31579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:02.970299 31579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:02.971678 31579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:02.971782 31579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:02.971879 31579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:02.971923 31579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:02.985634 31579 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:33645
I20260807 06:15:02.985658 31769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:33645 every 8 connection(s)
I20260807 06:15:02.987196 31579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:15:02.989651 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 31579
I20260807 06:15:02.990221 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:36105
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:32927
--webserver_port=38079
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:34417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:02.993126 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.990505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:43039)
0807 06:15:02.991633 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.991831 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":146,"threads_started":1}
W20260807 06:15:02.993801 31774 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:32927 (0 consecutive failures): Network error: Failed to ping master at 127.9.133.190:32927: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
I20260807 06:15:02.994786 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.994505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:52235)
0807 06:15:02.994654 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.994715 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 06:15:02.995651 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:02.995391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:33517)
0807 06:15:02.995529 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:02.995590 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 06:15:02.995848 31774 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:15:03.138972 31779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:03.139497 31779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:03.139591 31779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:03.144474 31779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:03.144593 31779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:03.144706 31779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:15:03.150720 31779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:36105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=38079
--tserver_master_addrs=127.9.133.190:32927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:03.152328 31779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:03.153851 31779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:03.166787 31790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:03.166919 31779 server_base.cc:1034] running on GCE node
W20260807 06:15:03.166965 31789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:03.166720 31792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:03.167902 31779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:03.168782 31779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:03.169974 31779 hybrid_clock.cc:648] HybridClock initialized: now 1786083303169921 us; error 63 us; skew 500 ppm
Aug 07 06:15:03 dist-test-slave-h3ft krb5kdc[29636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083303, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:03.173591 31779 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:15:03.174818 31779 webserver.cc:460] Webserver started at http://127.9.133.131:38079/ using document root <none> and password file <none>
I20260807 06:15:03.175590 31779 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:03.175666 31779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:03.182120 31779 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 06:15:03.185043 31802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:03.186916 31779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:15:03.187196 31779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "8584d84f032d4005903758430c8cff03"
format_stamp: "Formatted at 2026-08-07 06:15:00 on dist-test-slave-h3ft"
server_key: "4c4b034b3bb8a81c0dd6aec9e0ba1ca9"
server_key_iv: "6f98fa5f082ae2a239ea62b188c55d93"
server_key_version: "encryptionkey@0"
I20260807 06:15:03.188060 31779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:03.202438 31779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:03.206652 31779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:03.206962 31779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:03.207876 31779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:03.208976 31779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:03.209039 31779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:03.209121 31779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:03.209162 31779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:03.222981 31779 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:36105
I20260807 06:15:03.223091 31959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:36105 every 8 connection(s)
I20260807 06:15:03.224527 31779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:15:03.229851 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:03.227731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:59679)
0807 06:15:03.228673 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:03.228826 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":170,"threads_started":1}
W20260807 06:15:03.230659 31960 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:32927 (0 consecutive failures): Network error: Failed to ping master at 127.9.133.190:32927: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
I20260807 06:15:03.231513 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:03.231248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:54011)
0807 06:15:03.231380 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:03.231440 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 06:15:03.232342 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:03.232052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:36975)
0807 06:15:03.232213 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:03.232282 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 06:15:03.232645 31960 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:15:03.232795 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 31779
I20260807 06:15:03.768148 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:03.767523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:60285)
0807 06:15:03.767896 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:03.767977 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
I20260807 06:15:03.997308 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:03.996658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:35723)
0807 06:15:03.997056 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:03.997141 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":112,"threads_started":1}
I20260807 06:15:04.234160 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.233551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:52481)
0807 06:15:04.233907 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:04.233990 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":98,"threads_started":1}
I20260807 06:15:04.568408 31968 raft_consensus.cc:482] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:04.568701 31968 raft_consensus.cc:504] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:04.570341 31968 leader_election.cc:288] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
Aug 07 06:15:04 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.131@KRBTEST.COM
Aug 07 06:15:04 dist-test-slave-h3ft krb5kdc[29636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083302, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.130@KRBTEST.COM
I20260807 06:15:04.593185 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.570743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:36105 (local address 127.9.133.129:47513)
0807 06:15:04.571249 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:04.571269 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:04.571442 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:04.573918 (+ 2476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:04.573926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:04.574632 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:04.575561 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:04.575590 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.577866 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.577871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:04.579989 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:04.580006 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.581247 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.581252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:04.581590 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:04.582644 (+ 1054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:04.582680 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:04.585638 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:04.589737 (+ 4099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:04.589749 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:04.589767 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:04.590263 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:04.590762 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:04.590766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:04.590768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:04.590974 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:04.591866 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:04.591875 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:04.592190 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 06:15:04.592409 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 06:15:04.592914 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":483,"spinlock_wait_cycles":74624,"thread_start_us":86,"threads_started":1}
I20260807 06:15:04.593421 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.570963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:47513 (local address 127.9.133.131:36105)
0807 06:15:04.571575 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:04.571598 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:04.571653 (+ 55us) server_negotiation.cc:407] Connection header received
0807 06:15:04.572756 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:04.572766 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:04.573347 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:04.573759 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:04.575796 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.577619 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.580187 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.581032 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.581261 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:04.586181 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:04.586190 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:04.586217 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:04.586272 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:04.589496 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:04.590458 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:04.590465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:04.590470 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:04.590641 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:04.591164 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:04.591168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:04.591169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:04.591729 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:04.591902 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:04.592811 (+ 909us) server_negotiation.cc:299] Negotiation successful
0807 06:15:04.593171 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":162,"threads_started":1}
I20260807 06:15:04.593604 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.570964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:49627 (local address 127.9.133.130:33645)
0807 06:15:04.571575 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:04.571598 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:04.571653 (+ 55us) server_negotiation.cc:407] Connection header received
0807 06:15:04.572756 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:04.572766 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:04.573348 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:04.574096 (+ 748us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:04.575797 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.577620 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.580187 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.581076 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.581261 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:04.586180 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:04.586191 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:04.586217 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:04.586272 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:04.589497 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:04.590460 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:04.590467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:04.590470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:04.590641 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:04.591133 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:04.591137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:04.591139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:04.591731 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:04.591903 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:04.592947 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0807 06:15:04.593334 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":120,"threads_started":1}
I20260807 06:15:04.594164 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.570743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33645 (local address 127.9.133.129:49627)
0807 06:15:04.571249 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:04.571269 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:04.571442 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:04.573918 (+ 2476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:04.573926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:04.574657 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:04.575561 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:04.575590 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.577866 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.577871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:04.579989 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:04.580006 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:04.581247 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:04.581252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:04.581584 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:04.582644 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:04.582684 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:04.585761 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:04.589737 (+ 3976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:04.589749 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:04.589767 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:04.590263 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:04.590763 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:04.590766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:04.590768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:04.590965 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:04.591866 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:04.591875 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:04.592191 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 06:15:04.592406 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 06:15:04.593974 (+ 1568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":460,"spinlock_wait_cycles":94080,"thread_start_us":167,"threads_started":1}
W20260807 06:15:04.597806 31402 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
W20260807 06:15:04.598029 31408 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:04.598143 31408 leader_election.cc:302] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f51469ae9114174ad5b714981331150; no voters: 8584d84f032d4005903758430c8cff03, 9980c3aa9b8a4ace816a532c0ffa8800
I20260807 06:15:04.598440 31968 raft_consensus.cc:2726] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:15:04.769553 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.769191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:60607)
0807 06:15:04.769365 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:04.769432 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 06:15:04.999110 31977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:04.998469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:33589)
0807 06:15:04.998803 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:04.998868 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":106,"threads_started":1}
I20260807 06:15:05.235977 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:05.235396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:36183)
0807 06:15:05.235743 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:05.235820 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":107,"threads_started":1}
I20260807 06:15:05.771356 31979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:05.770677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:42377)
0807 06:15:05.771132 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:05.771214 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":127,"threads_started":1}
I20260807 06:15:06.000669 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:06.000064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:58101)
0807 06:15:06.000435 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:06.000525 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
I20260807 06:15:06.237685 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:06.237080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:56113)
0807 06:15:06.237448 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:06.237524 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
I20260807 06:15:06.399216 31985 raft_consensus.cc:482] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:06.399446 31985 raft_consensus.cc:504] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:06.400058 31985 leader_election.cc:288] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
W20260807 06:15:06.401245 31402 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
W20260807 06:15:06.401361 31408 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:06.401494 31408 leader_election.cc:302] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f51469ae9114174ad5b714981331150; no voters: 8584d84f032d4005903758430c8cff03, 9980c3aa9b8a4ace816a532c0ffa8800
I20260807 06:15:06.401688 31985 raft_consensus.cc:2726] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:15:06.773013 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:06.772365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:49333)
0807 06:15:06.772763 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:06.772838 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":149,"threads_started":1}
I20260807 06:15:07.002382 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:07.001775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:54385)
0807 06:15:07.002155 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:07.002237 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":114,"threads_started":1}
I20260807 06:15:07.239420 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:07.238787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:34857)
0807 06:15:07.239189 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:07.239258 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":143,"threads_started":1}
I20260807 06:15:07.774884 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:07.774271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:53067)
0807 06:15:07.774656 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:07.774732 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":97,"threads_started":1}
I20260807 06:15:08.004302 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:08.003656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:33019)
0807 06:15:08.004080 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:08.004162 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":140,"threads_started":1}
I20260807 06:15:08.241205 31993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:08.240573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:59181)
0807 06:15:08.240959 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:08.241067 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":151,"threads_started":1}
I20260807 06:15:08.534256 31996 raft_consensus.cc:482] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:08.534462 31996 raft_consensus.cc:504] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:08.535247 31996 leader_election.cc:288] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
W20260807 06:15:08.536507 31408 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
W20260807 06:15:08.536670 31402 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:08.536753 31402 leader_election.cc:302] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f51469ae9114174ad5b714981331150; no voters: 8584d84f032d4005903758430c8cff03, 9980c3aa9b8a4ace816a532c0ffa8800
I20260807 06:15:08.536928 31996 raft_consensus.cc:2726] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:15:08.777021 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:08.776285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:34123)
0807 06:15:08.776684 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:08.776833 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":112,"threads_started":1}
I20260807 06:15:09.005959 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:09.005331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:37769)
0807 06:15:09.005716 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:09.005798 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":146,"threads_started":1}
I20260807 06:15:09.242730 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:09.242189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:41785)
0807 06:15:09.242533 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:09.242603 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
I20260807 06:15:09.778537 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:09.778014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:50113)
0807 06:15:09.778321 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:09.778379 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":99,"threads_started":1}
I20260807 06:15:10.008038 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:10.007278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:44023)
0807 06:15:10.007741 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:10.007830 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":108,"threads_started":1}
I20260807 06:15:10.244451 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:10.243765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:34241)
0807 06:15:10.244219 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:10.244287 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":158,"threads_started":1}
I20260807 06:15:10.780122 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:10.779521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:40571)
0807 06:15:10.779865 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:10.779939 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260807 06:15:11.009935 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:11.009250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:36559)
0807 06:15:11.009680 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:11.009783 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
I20260807 06:15:11.246140 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:11.245504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:47749)
0807 06:15:11.245865 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:11.245947 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
I20260807 06:15:11.781699 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:11.781130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:49143)
0807 06:15:11.781496 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:11.781566 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":140,"threads_started":1}
I20260807 06:15:12.011785 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:12.011182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:50819)
0807 06:15:12.011564 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:12.011645 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":138,"threads_started":1}
I20260807 06:15:12.247890 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:12.247250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:47161)
0807 06:15:12.247662 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:12.247742 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":157,"threads_started":1}
I20260807 06:15:12.558482 32015 raft_consensus.cc:482] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:12.558682 32015 raft_consensus.cc:504] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9980c3aa9b8a4ace816a532c0ffa8800" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33645 } } peers { permanent_uuid: "8584d84f032d4005903758430c8cff03" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 36105 } } peers { permanent_uuid: "4f51469ae9114174ad5b714981331150" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42921 } }
I20260807 06:15:12.559259 32015 leader_election.cc:288] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645), 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105)
W20260807 06:15:12.560359 31408 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8584d84f032d4005903758430c8cff03 (127.9.133.131:36105): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
W20260807 06:15:12.560532 31402 leader_election.cc:341] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9980c3aa9b8a4ace816a532c0ffa8800 (127.9.133.130:33645): Not found: Tablet not found: 0b7bc99c4938414fac85b0ff4bb31600
I20260807 06:15:12.560611 31402 leader_election.cc:302] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f51469ae9114174ad5b714981331150; no voters: 8584d84f032d4005903758430c8cff03, 9980c3aa9b8a4ace816a532c0ffa8800
I20260807 06:15:12.560753 32015 raft_consensus.cc:2726] T 0b7bc99c4938414fac85b0ff4bb31600 P 4f51469ae9114174ad5b714981331150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:15:12.783550 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:12.782868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:56403)
0807 06:15:12.783271 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:12.783343 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":183,"threads_started":1}
I20260807 06:15:13.013478 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:13.012871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:50337)
0807 06:15:13.013262 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:13.013337 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":158,"threads_started":1}
I20260807 06:15:13.249356 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:13.248834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:55137)
0807 06:15:13.249158 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:13.249227 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":128,"threads_started":1}
I20260807 06:15:13.785107 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:13.784564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:58945)
0807 06:15:13.784891 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:13.784962 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":99,"threads_started":1}
I20260807 06:15:14.015286 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:14.014625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:48899)
0807 06:15:14.014990 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:14.015127 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":124,"threads_started":1}
I20260807 06:15:14.251143 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:14.250435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:44697)
0807 06:15:14.250840 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:14.250937 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":108,"threads_started":1}
I20260807 06:15:14.786801 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:14.786245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:59505)
0807 06:15:14.786592 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:14.786665 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":103,"threads_started":1}
I20260807 06:15:15.016830 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:15.016264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:51369)
0807 06:15:15.016618 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:15.016688 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260807 06:15:15.252774 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:15.252201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:38131)
0807 06:15:15.252547 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:15.252639 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":128,"threads_started":1}
I20260807 06:15:15.788609 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:15.788007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:33527)
0807 06:15:15.788389 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:15.788469 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":135,"threads_started":1}
I20260807 06:15:16.018677 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:16.018056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:51577)
0807 06:15:16.018426 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:16.018534 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":106,"threads_started":1}
I20260807 06:15:16.254601 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:16.254000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:34571)
0807 06:15:16.254358 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:16.254458 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260807 06:15:16.790278 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:16.789682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:36501)
0807 06:15:16.790069 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:16.790141 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":155,"threads_started":1}
I20260807 06:15:17.020529 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:17.019736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:48899)
0807 06:15:17.020126 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:17.020378 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260807 06:15:17.256233 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:17.255648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:59267)
0807 06:15:17.255986 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:17.256068 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
I20260807 06:15:17.792094 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:17.791546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.129:48391)
0807 06:15:17.791878 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:17.791948 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260807 06:15:18.022492 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:18.021766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.130:48511)
0807 06:15:18.022186 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:18.022291 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":134,"threads_started":1}
I20260807 06:15:18.242200 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 31332
I20260807 06:15:18.257061 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 31579
I20260807 06:15:18.262149 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:18.257445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:32927 (local address 127.9.133.131:46647)
0807 06:15:18.261853 (+ 4408us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:18.261958 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:32927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4265,"thread_start_us":101,"threads_started":1}
I20260807 06:15:18.269913 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 31779
2026-08-07T06:15:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20547 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:15:18 dist-test-slave-h3ft krb5kdc[32042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:15:18 dist-test-slave-h3ft krb5kdc[32042](info): set up 2 sockets
Aug 07 06:15:18 dist-test-slave-h3ft krb5kdc[32042](info): commencing operation
krb5kdc: starting...
W20260807 06:15:19.595494 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.004s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:15:19 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:15:19.754717 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40181
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:19.892601 32059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:19.892983 32059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:19.893070 32059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:19.897310 32059 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:15:19.897401 32059 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:15:19.897429 32059 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:15:19.897456 32059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:15:19.897512 32059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:19.897552 32059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:19.897591 32059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:15:19.897629 32059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:15:19.903550 32059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:19.905009 32059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:19.906353 32059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:19.917181 32066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:19.917181 32065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:19.917335 32059 server_base.cc:1034] running on GCE node
W20260807 06:15:19.917181 32068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:19.918259 32059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:19.919759 32059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:19.920993 32059 hybrid_clock.cc:648] HybridClock initialized: now 1786083319920965 us; error 45 us; skew 500 ppm
Aug 07 06:15:19 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083319, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:19.924746 32059 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:15:19.926029 32059 webserver.cc:460] Webserver started at http://127.9.133.190:36129/ using document root <none> and password file <none>
I20260807 06:15:19.926776 32059 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:19.926851 32059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:19.927165 32059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:15:19.930001 32059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "ffb1795312064068a6becd649c5b927c"
format_stamp: "Formatted at 2026-08-07 06:15:19 on dist-test-slave-h3ft"
server_key: "7fe2b56b25eb0b0d8626071786043b68"
server_key_iv: "15638947a6b60480f87141a29b348b17"
server_key_version: "encryptionkey@0"
I20260807 06:15:19.930744 32059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "ffb1795312064068a6becd649c5b927c"
format_stamp: "Formatted at 2026-08-07 06:15:19 on dist-test-slave-h3ft"
server_key: "7fe2b56b25eb0b0d8626071786043b68"
server_key_iv: "15638947a6b60480f87141a29b348b17"
server_key_version: "encryptionkey@0"
I20260807 06:15:19.935994 32059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 06:15:19.939566 32078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:19.941354 32059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:15:19.941568 32059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "ffb1795312064068a6becd649c5b927c"
format_stamp: "Formatted at 2026-08-07 06:15:19 on dist-test-slave-h3ft"
server_key: "7fe2b56b25eb0b0d8626071786043b68"
server_key_iv: "15638947a6b60480f87141a29b348b17"
server_key_version: "encryptionkey@0"
I20260807 06:15:19.941741 32059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:19.957476 32059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:19.961686 32059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:19.961972 32059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:19.970875 32059 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40181
I20260807 06:15:19.971284 32165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40181 every 8 connection(s)
I20260807 06:15:19.972219 32059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:15:19.975404 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 32059
I20260807 06:15:19.975574 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:15:19.975929 9750 external_mini_cluster.cc:1428] Setting key 55c89f410fc12127ac0c2d3dac2e1142
I20260807 06:15:19.976135 32167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:19.985517 32167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap starting.
Aug 07 06:15:19 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:19.988703 32167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:19.989687 32167 log.cc:824] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:19.992684 32167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: No bootstrap required, opened a new log
I20260807 06:15:19.993958 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:19.977802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47244 (local address 127.9.133.190:40181)
0807 06:15:19.978340 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:19.978363 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:19.978418 (+ 55us) server_negotiation.cc:407] Connection header received
0807 06:15:19.979730 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:19.979766 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:19.980311 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:19.980783 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:19.981615 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:19.983280 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:19.984242 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:19.984975 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:19.985152 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:19.988136 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:19.988143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:19.988163 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:19.988202 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:19.990501 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:19.991144 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:19.991149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:19.991155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:19.991290 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:19.991607 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:19.991610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:19.991611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:19.992101 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:19.992280 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:19.992634 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 06:15:19.993009 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":160,"threads_started":1}
I20260807 06:15:19.997437 32167 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:19.997696 32167 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:19.997779 32167 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb1795312064068a6becd649c5b927c, State: Initialized, Role: FOLLOWER
I20260807 06:15:19.998268 32167 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:19.998431 32167 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:15:19.998498 32167 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:15:19.998612 32167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:19.999864 32167 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:20.000303 32167 leader_election.cc:302] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffb1795312064068a6becd649c5b927c; no voters:
I20260807 06:15:20.000725 32167 leader_election.cc:288] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:15:20.000888 32174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:15:20.001225 32174 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 LEADER]: Becoming Leader. State: Replica: ffb1795312064068a6becd649c5b927c, State: Running, Role: LEADER
I20260807 06:15:20.001680 32174 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:20.002439 32167 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:15:20.004014 32178 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffb1795312064068a6becd649c5b927c. Latest consensus state: current_term: 1 leader_uuid: "ffb1795312064068a6becd649c5b927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } } }
I20260807 06:15:20.004199 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffb1795312064068a6becd649c5b927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } } }
I20260807 06:15:20.004254 32178 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:20.004323 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:20.004925 32194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:15:20.009797 32194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:15:20.018143 32194 catalog_manager.cc:1347] Generated new cluster ID: 32c2ca61d52844efac834b17f38dc473
I20260807 06:15:20.018290 32194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:15:20.042742 32194 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:15:20.044487 32194 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:15:20.054966 32194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Generated new TSK 0
I20260807 06:15:20.056159 32194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:15:20.112494 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40181
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:20.273028 32210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:20.273339 32210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:20.273412 32210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:20.277588 32210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:20.277665 32210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:20.277727 32210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:15:20.283578 32210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:20.284972 32210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:20.286281 32210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:20.298195 32216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:20.298187 32218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:20.298187 32215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:20.298557 32210 server_base.cc:1034] running on GCE node
I20260807 06:15:20.299109 32210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:20.299938 32210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:20.301120 32210 hybrid_clock.cc:648] HybridClock initialized: now 1786083320301102 us; error 36 us; skew 500 ppm
Aug 07 06:15:20 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083320, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:20.304921 32210 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:15:20.306075 32210 webserver.cc:460] Webserver started at http://127.9.133.129:40237/ using document root <none> and password file <none>
I20260807 06:15:20.306792 32210 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:20.306862 32210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:20.307192 32210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:15:20.309811 32210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "55b04726cc014061b197d248cbdd2c79"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "182d8693c7a841b47bc6fa7e70fc791d"
server_key_iv: "c343bb48c90a81cc89ff6fc84444bbfa"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.310479 32210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "55b04726cc014061b197d248cbdd2c79"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "182d8693c7a841b47bc6fa7e70fc791d"
server_key_iv: "c343bb48c90a81cc89ff6fc84444bbfa"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.315128 32210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260807 06:15:20.318015 32228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.319712 32210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:20.319880 32210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "55b04726cc014061b197d248cbdd2c79"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "182d8693c7a841b47bc6fa7e70fc791d"
server_key_iv: "c343bb48c90a81cc89ff6fc84444bbfa"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.320060 32210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:20.344338 32210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:20.348899 32210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:20.349179 32210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:20.350031 32210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:20.351228 32210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:20.351311 32210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.351397 32210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:20.351455 32210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.363363 32210 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:38615
I20260807 06:15:20.363397 32416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:38615 every 8 connection(s)
I20260807 06:15:20.364513 32210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:15:20.374406 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 32210
I20260807 06:15:20.374570 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:15:20.374953 9750 external_mini_cluster.cc:1428] Setting key 3207acb9ed826b9e51ecd0545ad65337
Aug 07 06:15:20 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083320, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:20.380803 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.366906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:40339 (local address 127.9.133.190:40181)
0807 06:15:20.367152 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.367158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.368486 (+ 1328us) server_negotiation.cc:407] Connection header received
0807 06:15:20.369494 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.369498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.369559 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.369648 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:20.371058 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.371735 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.373031 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.373319 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.373433 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.376691 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:20.376695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:20.376698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:20.376726 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:20.378481 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.379085 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.379089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.379091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.379189 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.379575 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.379578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.379581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.379786 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:20.379904 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.380460 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.380603 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 06:15:20.381763 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.367461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:40339)
0807 06:15:20.368302 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:20.368351 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:20.369265 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:20.369848 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:20.369855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:20.370310 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:20.370872 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.370887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.371859 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.371862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.372896 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.372907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.373487 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.373489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.373643 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.374334 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:20.374358 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:20.376512 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:20.378619 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:20.378625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:20.378642 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:20.378938 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:20.379296 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:20.379299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:20.379301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:20.379470 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:20.379904 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:20.379912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:20.380194 (+ 282us) client_negotiation.cc:769] Sending connection context
0807 06:15:20.380390 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 06:15:20.380755 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":149,"threads_started":1}
I20260807 06:15:20.383407 32417 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40181
I20260807 06:15:20.383695 32417 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:20.384274 32417 heartbeater.cc:511] Master 127.9.133.190:40181 requested a full tablet report, sending...
I20260807 06:15:20.386303 32108 ts_manager.cc:194] Registered new tserver with Master: 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615)
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:15:20.388317 32108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:40339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:15:20.411437 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40181
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:20.546463 32424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:20.546789 32424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:20.546834 32424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:20.551105 32424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:20.551188 32424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:20.551256 32424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:15:20.557129 32424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:20.558449 32424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:20.559841 32424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:20.571487 32435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:20.571568 32432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:20.571597 32431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:20.572072 32424 server_base.cc:1034] running on GCE node
I20260807 06:15:20.572567 32424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:20.573366 32424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:20.574595 32424 hybrid_clock.cc:648] HybridClock initialized: now 1786083320574553 us; error 56 us; skew 500 ppm
Aug 07 06:15:20 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083320, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:20.578284 32424 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:15:20.579682 32424 webserver.cc:460] Webserver started at http://127.9.133.130:43491/ using document root <none> and password file <none>
I20260807 06:15:20.580463 32424 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:20.580576 32424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:20.580878 32424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:15:20.583637 32424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "9df1009cda874613985fdb2c47731afe"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "ef72359aaef673a211a306a87fcfdd95"
server_key_iv: "1ebc0da8400013a65a65132f07c472b7"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.584367 32424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "9df1009cda874613985fdb2c47731afe"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "ef72359aaef673a211a306a87fcfdd95"
server_key_iv: "1ebc0da8400013a65a65132f07c472b7"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.589227 32424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 06:15:20.592381 32446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.594017 32424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:15:20.594184 32424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "9df1009cda874613985fdb2c47731afe"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "ef72359aaef673a211a306a87fcfdd95"
server_key_iv: "1ebc0da8400013a65a65132f07c472b7"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.594378 32424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:20.618022 32424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:20.622438 32424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:20.622701 32424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:20.623533 32424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:20.624627 32424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:20.624707 32424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.624814 32424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:20.624840 32424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.636031 32424 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:37927
I20260807 06:15:20.636055 32624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:37927 every 8 connection(s)
I20260807 06:15:20.637173 32424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:15:20.641196 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 32424
I20260807 06:15:20.641357 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:15:20.641680 9750 external_mini_cluster.cc:1428] Setting key c5581fb084dc59883b892c8255e5f7bf
Aug 07 06:15:20 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083320, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:20.653518 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.639555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:55951 (local address 127.9.133.190:40181)
0807 06:15:20.639772 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.639776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.641032 (+ 1256us) server_negotiation.cc:407] Connection header received
0807 06:15:20.642306 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.642310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.642386 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.642504 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:20.643996 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.644594 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.645926 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.646179 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.646311 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.649248 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:20.649253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:20.649256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:20.649291 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:20.651107 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.651655 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.651659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.651661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.651754 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.652207 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.652210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.652212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.652439 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:20.652567 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.653144 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.653306 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:15:20.654572 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.640018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:55951)
0807 06:15:20.640829 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:20.640877 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:20.642043 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:20.642719 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:20.642727 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:20.643248 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:20.643835 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.643850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.644743 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.644747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.645786 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.645797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.646374 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.646378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.646611 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.647393 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:20.647411 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:20.649065 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:20.651228 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:20.651235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:20.651253 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:20.651542 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:20.651852 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:20.651856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:20.651858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:20.652096 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:20.652562 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:20.652569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:20.652879 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 06:15:20.653101 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 06:15:20.653481 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":139,"threads_started":1}
I20260807 06:15:20.656026 32625 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40181
I20260807 06:15:20.656327 32625 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:15:20.656953 32625 heartbeater.cc:511] Master 127.9.133.190:40181 requested a full tablet report, sending...
I20260807 06:15:20.658243 32108 ts_manager.cc:194] Registered new tserver with Master: 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
I20260807 06:15:20.658936 32108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:55951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:15:20.678756 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40181
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:20.816241 32632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:20.816563 32632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:20.816610 32632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:20.821509 32632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:20.821580 32632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:20.821641 32632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:15:20.827605 32632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:20.828893 32632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:20.830166 32632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:20.842401 32647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:20.842377 32638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:20.842377 32640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:20.843138 32632 server_base.cc:1034] running on GCE node
I20260807 06:15:20.843866 32632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:20.845098 32632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:20.846477 32632 hybrid_clock.cc:648] HybridClock initialized: now 1786083320846448 us; error 46 us; skew 500 ppm
Aug 07 06:15:20 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083320, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:20.851089 32632 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:15:20.852639 32632 webserver.cc:460] Webserver started at http://127.9.133.131:33115/ using document root <none> and password file <none>
I20260807 06:15:20.853611 32632 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:20.853698 32632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:20.853973 32632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:15:20.856657 32632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "02523226508346e98a2d555054c618ab"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "5d4195c68f451bb5d0b77e42541eb8fe"
server_key_iv: "45bcd398c017cf386133aa73746e08a4"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.857329 32632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "02523226508346e98a2d555054c618ab"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "5d4195c68f451bb5d0b77e42541eb8fe"
server_key_iv: "45bcd398c017cf386133aa73746e08a4"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.862028 32632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 06:15:20.865145 32660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.866819 32632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:15:20.867103 32632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "02523226508346e98a2d555054c618ab"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "5d4195c68f451bb5d0b77e42541eb8fe"
server_key_iv: "45bcd398c017cf386133aa73746e08a4"
server_key_version: "encryptionkey@0"
I20260807 06:15:20.867260 32632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:20.881204 32632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:20.884783 32632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:20.885025 32632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:20.885751 32632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:20.886811 32632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:20.886891 32632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:20.886971 32632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:20.887058 32632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 06:15:20.897554 32632 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:42227
I20260807 06:15:20.897629 361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:42227 every 8 connection(s)
I20260807 06:15:20.898721 32632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:15:20.908839 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 32632
I20260807 06:15:20.909058 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:15:20.909493 9750 external_mini_cluster.cc:1428] Setting key 776bbfeca56f319ffa9d54687e3492d4
Aug 07 06:15:20 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083320, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:20.916206 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.901259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:44213 (local address 127.9.133.190:40181)
0807 06:15:20.901475 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.901479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.902664 (+ 1185us) server_negotiation.cc:407] Connection header received
0807 06:15:20.904470 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.904474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.904554 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.904676 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:20.906124 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.906747 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.908097 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.908428 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.908569 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.911994 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:20.911998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:20.912001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:20.912041 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:20.913729 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.914348 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.914359 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.914362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.914487 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.914877 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.914880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.914882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.915169 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:20.915279 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.915873 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.916018 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 06:15:20.917232 369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.901742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:44213)
0807 06:15:20.902491 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:20.902540 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:20.904210 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:20.904885 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:20.904893 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:20.905380 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:20.905962 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.905976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.906911 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.906919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.907954 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.907964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.908605 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.908610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.908819 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.909670 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:20.909689 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:20.911797 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:20.913874 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:20.913880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:20.913899 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:20.914202 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:20.914597 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:20.914600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:20.914603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:20.914766 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:20.915288 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:20.915298 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:20.915596 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 06:15:20.915816 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 06:15:20.916213 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":150,"threads_started":1}
I20260807 06:15:20.918669 363 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40181
I20260807 06:15:20.918957 363 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:20.919641 363 heartbeater.cc:511] Master 127.9.133.190:40181 requested a full tablet report, sending...
I20260807 06:15:20.920943 32108 ts_manager.cc:194] Registered new tserver with Master: 02523226508346e98a2d555054c618ab (127.9.133.131:42227)
I20260807 06:15:20.921612 32108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:44213
I20260807 06:15:20.924508 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:15:20.935858 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.927045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.9.133.190:40181)
0807 06:15:20.927189 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.927193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.927441 (+ 248us) server_negotiation.cc:407] Connection header received
0807 06:15:20.927526 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.927529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.927577 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.927650 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:20.928453 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.929272 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.930245 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.930477 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.930572 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.932427 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:20.932430 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:20.932432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:20.932457 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:20.934130 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.934549 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.934552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.934553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.934632 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:20.934867 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:20.934869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:20.934870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:20.935234 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:20.935350 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.935605 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.935711 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:15:20.939446 32108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:15:20.943300 32108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:15:20.961097 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.954745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.9.133.129:38615)
0807 06:15:20.955499 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.955510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.955540 (+ 30us) server_negotiation.cc:407] Connection header received
0807 06:15:20.955639 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.955646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.955912 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.956112 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:20.956473 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.957725 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.959358 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.960266 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.960395 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.960591 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.960760 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.960876 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":114,"threads_started":1}
I20260807 06:15:20.961409 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.955303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:42227 (local address 127.0.0.1:50590)
0807 06:15:20.955688 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:20.955702 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:20.955785 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:20.956940 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:20.956943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:20.956963 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:20.957088 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.957095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.958560 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.958564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.959668 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.959676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.960897 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.960900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.960997 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.961023 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:15:20.961104 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:15:20.961229 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":1,"thread_start_us":135,"threads_started":1}
I20260807 06:15:20.961567 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.955442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:37927 (local address 127.0.0.1:38326)
0807 06:15:20.955900 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:20.955917 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:20.956017 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:20.957270 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:20.957273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:20.957307 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:20.957397 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.957405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.958850 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.958853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.959993 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.960006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.961159 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.961163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.961244 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.961276 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 06:15:20.961325 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 06:15:20.961382 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":107,"threads_started":1}
I20260807 06:15:20.961885 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.954513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38615 (local address 127.0.0.1:50852)
0807 06:15:20.954989 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:20.955101 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:20.955346 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:20.956095 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:20.956103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:20.956128 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:20.956337 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.956347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.957858 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.957861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.959227 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:20.959239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.960395 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.960399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:20.960536 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.960579 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 06:15:20.960723 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 06:15:20.961699 (+ 976us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":43,"thread_start_us":147,"threads_started":1}
I20260807 06:15:20.962088 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.955962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.9.133.131:42227)
0807 06:15:20.956517 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.956538 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.956562 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:15:20.956639 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.956642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.956798 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.956973 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:20.957203 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.958392 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.959792 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.960567 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.960676 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.960824 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.961814 (+ 990us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.961928 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":205,"threads_started":1}
I20260807 06:15:20.962199 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:20.956277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.9.133.130:37927)
0807 06:15:20.956825 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:20.956838 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:20.956863 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:15:20.956949 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:20.956954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:20.957124 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:20.957291 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:20.957566 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.958697 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.960151 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:20.961026 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:20.961707 (+ 681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:20.961874 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:20.961922 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 06:15:20.962028 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":151,"threads_started":1}
I20260807 06:15:20.965901 32311 tablet_service.cc:1467] Processing CreateTablet for tablet 451918ba4896423a95ddb3c1fa3b019a (DEFAULT_TABLE table=test-table [id=e56d3a7bfc834ab281965acc1db17b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:20.965904 32730 tablet_service.cc:1467] Processing CreateTablet for tablet 451918ba4896423a95ddb3c1fa3b019a (DEFAULT_TABLE table=test-table [id=e56d3a7bfc834ab281965acc1db17b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:20.965904 32521 tablet_service.cc:1467] Processing CreateTablet for tablet 451918ba4896423a95ddb3c1fa3b019a (DEFAULT_TABLE table=test-table [id=e56d3a7bfc834ab281965acc1db17b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:20.967222 32521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451918ba4896423a95ddb3c1fa3b019a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:20.967222 32311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451918ba4896423a95ddb3c1fa3b019a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:20.967228 32730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451918ba4896423a95ddb3c1fa3b019a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:20.976226 388 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Bootstrap starting.
I20260807 06:15:20.977152 391 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Bootstrap starting.
I20260807 06:15:20.979173 388 tablet_bootstrap.cc:654] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:20.979444 389 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Bootstrap starting.
I20260807 06:15:20.979712 391 tablet_bootstrap.cc:654] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:20.980398 388 log.cc:824] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:20.980688 391 log.cc:824] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:20.981865 389 tablet_bootstrap.cc:654] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:20.983501 389 log.cc:824] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:20.984074 388 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: No bootstrap required, opened a new log
I20260807 06:15:20.984367 388 ts_tablet_manager.cc:1397] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260807 06:15:20.984378 391 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: No bootstrap required, opened a new log
I20260807 06:15:20.984563 391 ts_tablet_manager.cc:1397] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260807 06:15:20.988373 389 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: No bootstrap required, opened a new log
I20260807 06:15:20.988581 389 ts_tablet_manager.cc:1397] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 06:15:20.988561 391 raft_consensus.cc:348] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:20.988700 388 raft_consensus.cc:348] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:20.988897 391 raft_consensus.cc:374] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:20.988973 391 raft_consensus.cc:729] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df1009cda874613985fdb2c47731afe, State: Initialized, Role: FOLLOWER
I20260807 06:15:20.989002 388 raft_consensus.cc:374] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:20.989059 388 raft_consensus.cc:729] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b04726cc014061b197d248cbdd2c79, State: Initialized, Role: FOLLOWER
I20260807 06:15:20.989538 391 consensus_queue.cc:260] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:20.989559 388 consensus_queue.cc:260] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:20.990578 32417 heartbeater.cc:503] Master 127.9.133.190:40181 was elected leader, sending a full tablet report...
I20260807 06:15:20.990697 391 ts_tablet_manager.cc:1428] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260807 06:15:20.991117 388 ts_tablet_manager.cc:1428] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 06:15:20.991776 32625 heartbeater.cc:503] Master 127.9.133.190:40181 was elected leader, sending a full tablet report...
I20260807 06:15:20.992676 389 raft_consensus.cc:348] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:20.992993 389 raft_consensus.cc:374] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:20.993111 389 raft_consensus.cc:729] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02523226508346e98a2d555054c618ab, State: Initialized, Role: FOLLOWER
I20260807 06:15:20.993580 389 consensus_queue.cc:260] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:20.994333 363 heartbeater.cc:503] Master 127.9.133.190:40181 was elected leader, sending a full tablet report...
I20260807 06:15:20.994614 389 ts_tablet_manager.cc:1428] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 06:15:21.116333 32418 tablet.cc:2369] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:15:21.138616 32627 tablet.cc:2369] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:15:21.149482 399 raft_consensus.cc:482] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:21.149878 399 raft_consensus.cc:504] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
W20260807 06:15:21.149996 364 tablet.cc:2369] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:15:21.151585 399 leader_election.cc:288] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615), 02523226508346e98a2d555054c618ab (127.9.133.131:42227)
I20260807 06:15:21.156539 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:21.151921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38615 (local address 127.9.133.130:32929)
0807 06:15:21.152128 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:21.152171 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:21.152301 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:21.152720 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:21.152722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:21.152743 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:21.152907 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:21.152916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.153969 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.153973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:21.154960 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:21.154970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.155950 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.155953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:21.156075 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:21.156105 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 06:15:21.156198 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:15:21.156300 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":1}
I20260807 06:15:21.156850 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:21.151986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:42227 (local address 127.9.133.130:53247)
0807 06:15:21.152729 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:21.152747 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:21.152865 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:21.153200 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:21.153204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:21.153243 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:21.153410 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:21.153419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.154397 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.154400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:21.155451 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:21.155463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.156273 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.156280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:21.156407 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:21.156444 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 06:15:21.156542 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:15:21.156642 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":349,"thread_start_us":133,"threads_started":1}
I20260807 06:15:21.156924 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:21.152239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:53247 (local address 127.9.133.131:42227)
0807 06:15:21.152408 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:21.152415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:21.152864 (+ 449us) server_negotiation.cc:407] Connection header received
0807 06:15:21.152977 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:21.152982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:21.153048 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:21.153141 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:21.153585 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.154248 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.155599 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.156129 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.156244 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:21.156295 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:21.156592 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 06:15:21.156753 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:15:21.157573 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:21.152089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:32929 (local address 127.9.133.129:38615)
0807 06:15:21.152269 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:21.152275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:21.152296 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:15:21.152483 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:21.152486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:21.152588 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:21.152688 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:21.153069 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.153823 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.155168 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.155781 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.155901 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:21.155957 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:21.157265 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0807 06:15:21.157404 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 06:15:21.157910 32760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451918ba4896423a95ddb3c1fa3b019a" candidate_uuid: "9df1009cda874613985fdb2c47731afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02523226508346e98a2d555054c618ab" is_pre_election: true
I20260807 06:15:21.158278 32344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451918ba4896423a95ddb3c1fa3b019a" candidate_uuid: "9df1009cda874613985fdb2c47731afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55b04726cc014061b197d248cbdd2c79" is_pre_election: true
I20260807 06:15:21.158327 32760 raft_consensus.cc:2454] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9df1009cda874613985fdb2c47731afe in term 0.
I20260807 06:15:21.158612 32344 raft_consensus.cc:2454] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9df1009cda874613985fdb2c47731afe in term 0.
I20260807 06:15:21.158953 32450 leader_election.cc:302] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02523226508346e98a2d555054c618ab, 9df1009cda874613985fdb2c47731afe; no voters:
I20260807 06:15:21.159395 399 raft_consensus.cc:2781] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:15:21.159482 399 raft_consensus.cc:482] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:15:21.159512 399 raft_consensus.cc:3037] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:21.160810 399 raft_consensus.cc:504] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:21.161238 399 leader_election.cc:288] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [CANDIDATE]: Term 1 election: Requested vote from peers 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615), 02523226508346e98a2d555054c618ab (127.9.133.131:42227)
I20260807 06:15:21.161629 32344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451918ba4896423a95ddb3c1fa3b019a" candidate_uuid: "9df1009cda874613985fdb2c47731afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55b04726cc014061b197d248cbdd2c79"
I20260807 06:15:21.161764 32344 raft_consensus.cc:3037] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:21.161792 32760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451918ba4896423a95ddb3c1fa3b019a" candidate_uuid: "9df1009cda874613985fdb2c47731afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02523226508346e98a2d555054c618ab"
I20260807 06:15:21.161991 32760 raft_consensus.cc:3037] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:21.163195 32344 raft_consensus.cc:2454] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9df1009cda874613985fdb2c47731afe in term 1.
I20260807 06:15:21.163285 32760 raft_consensus.cc:2454] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9df1009cda874613985fdb2c47731afe in term 1.
I20260807 06:15:21.163605 32449 leader_election.cc:302] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55b04726cc014061b197d248cbdd2c79, 9df1009cda874613985fdb2c47731afe; no voters:
I20260807 06:15:21.163798 399 raft_consensus.cc:2781] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:15:21.164111 399 raft_consensus.cc:686] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 1 LEADER]: Becoming Leader. State: Replica: 9df1009cda874613985fdb2c47731afe, State: Running, Role: LEADER
I20260807 06:15:21.164562 399 consensus_queue.cc:237] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:21.168926 32104 catalog_manager.cc:5495] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe reported cstate change: term changed from 0 to 1, leader changed from <none> to 9df1009cda874613985fdb2c47731afe (127.9.133.130). New cstate: current_term: 1 leader_uuid: "9df1009cda874613985fdb2c47731afe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } health_report { overall_health: UNKNOWN } } }
I20260807 06:15:21.200814 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:21.195942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.9.133.130:37927)
0807 06:15:21.196101 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:21.196106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:21.196123 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:15:21.196335 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:21.196339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:21.196413 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:21.196512 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:15:21.196958 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.197536 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.198806 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:21.199244 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:21.199371 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:21.199809 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:15:21.200380 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:15:21.200528 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:21.200565 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 06:15:21.200637 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 06:15:21.208923 32760 raft_consensus.cc:1261] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Refusing update from remote peer 9df1009cda874613985fdb2c47731afe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:21.208923 32344 raft_consensus.cc:1261] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Refusing update from remote peer 9df1009cda874613985fdb2c47731afe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:21.209833 402 consensus_queue.cc:1035] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [LEADER]: Connected to new peer: Peer: permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:21.210175 399 consensus_queue.cc:1035] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [LEADER]: Connected to new peer: Peer: permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:21.229728 32104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:21.229971 32104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:21.233147 32104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Sending DeleteTablet for 3 replicas of tablet 451918ba4896423a95ddb3c1fa3b019a
I20260807 06:15:21.234200 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 32210
I20260807 06:15:21.234211 32311 tablet_service.cc:1514] Processing DeleteTablet for tablet 451918ba4896423a95ddb3c1fa3b019a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:50852
I20260807 06:15:21.235049 32730 tablet_service.cc:1514] Processing DeleteTablet for tablet 451918ba4896423a95ddb3c1fa3b019a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:50590
I20260807 06:15:21.241636 419 tablet_replica.cc:330] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: stopping tablet replica
I20260807 06:15:21.242060 32521 tablet_service.cc:1514] Processing DeleteTablet for tablet 451918ba4896423a95ddb3c1fa3b019a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:38326
I20260807 06:15:21.242357 419 raft_consensus.cc:2229] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:21.242784 421 tablet_replica.cc:330] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: stopping tablet replica
I20260807 06:15:21.243042 419 raft_consensus.cc:2258] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:21.243173 421 raft_consensus.cc:2229] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:15:21.243690 421 raft_consensus.cc:2258] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:21.246222 421 ts_tablet_manager.cc:1905] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:21.248762 419 ts_tablet_manager.cc:1905] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:21.249821 421 ts_tablet_manager.cc:1918] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:21.249959 421 log.cc:1196] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/451918ba4896423a95ddb3c1fa3b019a
I20260807 06:15:21.250327 421 ts_tablet_manager.cc:1939] T 451918ba4896423a95ddb3c1fa3b019a P 9df1009cda874613985fdb2c47731afe: Deleting consensus metadata
I20260807 06:15:21.252071 32080 catalog_manager.cc:4841] TS 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927): tablet 451918ba4896423a95ddb3c1fa3b019a (table test-table [id=e56d3a7bfc834ab281965acc1db17b18]) successfully deleted
I20260807 06:15:21.252503 419 ts_tablet_manager.cc:1918] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:21.252658 419 log.cc:1196] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/451918ba4896423a95ddb3c1fa3b019a
I20260807 06:15:21.253021 419 ts_tablet_manager.cc:1939] T 451918ba4896423a95ddb3c1fa3b019a P 02523226508346e98a2d555054c618ab: Deleting consensus metadata
I20260807 06:15:21.254448 32081 catalog_manager.cc:4841] TS 02523226508346e98a2d555054c618ab (127.9.133.131:42227): tablet 451918ba4896423a95ddb3c1fa3b019a (table test-table [id=e56d3a7bfc834ab281965acc1db17b18]) successfully deleted
W20260807 06:15:21.255327 32080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:38615 (error 108)
I20260807 06:15:21.256515 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 32424
I20260807 06:15:21.256644 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:21.256280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38615 (local address 127.0.0.1:50864)
0807 06:15:21.256431 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:21.256516 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 06:15:21.260257 32080 catalog_manager.cc:4584] TS 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 451918ba4896423a95ddb3c1fa3b019a: Network error: Client connection negotiation failed: client connection to 127.9.133.129:38615: connect: Connection refused (error 111)
I20260807 06:15:21.274972 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 32632
I20260807 06:15:21.290577 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 32059
I20260807 06:15:41.307148 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40181
--webserver_interface=127.9.133.190
--webserver_port=36129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:15:41.449285 424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:41.449610 424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:41.449654 424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:41.454137 424 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:15:41.454205 424 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:15:41.454221 424 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:15:41.454236 424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:15:41.454254 424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:41.454273 424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:41.454347 424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:15:41.454380 424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:15:41.460371 424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=36129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:41.461812 424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:41.463316 424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:41.481299 435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:41.481339 424 server_base.cc:1034] running on GCE node
W20260807 06:15:41.481299 438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:41.481343 433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:41.482520 424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:41.484143 424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:41.485348 424 hybrid_clock.cc:648] HybridClock initialized: now 1786083341485300 us; error 61 us; skew 500 ppm
Aug 07 06:15:41 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083341, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:41.488885 424 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:15:41.490057 424 webserver.cc:460] Webserver started at http://127.9.133.190:36129/ using document root <none> and password file <none>
I20260807 06:15:41.490748 424 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:41.490829 424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:41.495651 424 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:15:41.498134 448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:41.499819 424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:41.500001 424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "ffb1795312064068a6becd649c5b927c"
format_stamp: "Formatted at 2026-08-07 06:15:19 on dist-test-slave-h3ft"
server_key: "7fe2b56b25eb0b0d8626071786043b68"
server_key_iv: "15638947a6b60480f87141a29b348b17"
server_key_version: "encryptionkey@0"
I20260807 06:15:41.500754 424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:41.531518 424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:41.536116 424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:41.536427 424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:41.546098 424 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40181
I20260807 06:15:41.546558 519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40181 every 8 connection(s)
I20260807 06:15:41.547621 424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:15:41.548679 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 424
I20260807 06:15:41.549335 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:38615
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:40181
--webserver_port=40237
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:41.554683 520 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:15:41.556936 520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap starting.
I20260807 06:15:41.579331 520 log.cc:824] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:41.590509 520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:15:41.591125 520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap complete.
I20260807 06:15:41.596184 520 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:41.596671 520 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb1795312064068a6becd649c5b927c, State: Initialized, Role: FOLLOWER
I20260807 06:15:41.597342 520 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:41.597488 520 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:15:41.597599 520 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:15:41.597733 520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 1 FOLLOWER]: Advancing to term 2
I20260807 06:15:41.600292 520 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:41.600759 520 leader_election.cc:302] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffb1795312064068a6becd649c5b927c; no voters:
I20260807 06:15:41.601235 520 leader_election.cc:288] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 06:15:41.601365 525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Leader election won for term 2
I20260807 06:15:41.601639 525 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 LEADER]: Becoming Leader. State: Replica: ffb1795312064068a6becd649c5b927c, State: Running, Role: LEADER
I20260807 06:15:41.602174 525 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:15:41.602931 520 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:15:41.604147 530 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffb1795312064068a6becd649c5b927c. Latest consensus state: current_term: 2 leader_uuid: "ffb1795312064068a6becd649c5b927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } } }
I20260807 06:15:41.604298 530 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:41.604642 529 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ffb1795312064068a6becd649c5b927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } } }
I20260807 06:15:41.604738 529 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: This master's current role is: LEADER
I20260807 06:15:41.606568 542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:15:41.612958 542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:15:41.613756 542 catalog_manager.cc:1259] Loaded cluster ID: 32c2ca61d52844efac834b17f38dc473
I20260807 06:15:41.613852 542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:15:41.616870 542 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:15:41.617746 542 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Loaded TSK: 0
I20260807 06:15:41.636620 542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Generated new TSK 1
I20260807 06:15:41.637032 542 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:15:41.694111 522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:41.694530 522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:41.694623 522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:41.699236 522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:41.699343 522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:41.699445 522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:15:41.706614 522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:38615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=40237
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:41.708165 522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:41.709688 522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:41.721840 561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:41.721972 567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:41.721840 562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:41.722218 522 server_base.cc:1034] running on GCE node
I20260807 06:15:41.722958 522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:41.723951 522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:41.725167 522 hybrid_clock.cc:648] HybridClock initialized: now 1786083341725130 us; error 49 us; skew 500 ppm
Aug 07 06:15:41 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083341, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:41.729494 522 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:15:41.730924 522 webserver.cc:460] Webserver started at http://127.9.133.129:40237/ using document root <none> and password file <none>
I20260807 06:15:41.731815 522 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:41.731946 522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:41.738317 522 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:15:41.741798 582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:41.743791 522 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:15:41.743952 522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "55b04726cc014061b197d248cbdd2c79"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "182d8693c7a841b47bc6fa7e70fc791d"
server_key_iv: "c343bb48c90a81cc89ff6fc84444bbfa"
server_key_version: "encryptionkey@0"
I20260807 06:15:41.744856 522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:41.783769 522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:41.788527 522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:41.788812 522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:41.789808 522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:41.791414 590 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:15:41.797003 522 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:15:41.797109 522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 06:15:41.797227 522 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:15:41.799615 522 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:15:41.799695 522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260807 06:15:41.800115 590 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Bootstrap starting.
I20260807 06:15:41.813941 522 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:38615
I20260807 06:15:41.814021 758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:38615 every 8 connection(s)
I20260807 06:15:41.815338 522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:15:41.820955 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 522
I20260807 06:15:41.821558 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:37927
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:40181
--webserver_port=43491
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:15:41.824344 590 log.cc:824] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:15:41 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083341, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:41.836681 590 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:15:41.837301 590 tablet_bootstrap.cc:492] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Bootstrap complete.
I20260807 06:15:41.837697 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:41.818383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:42161)
0807 06:15:41.819414 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:41.819503 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:41.820891 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:41.822831 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:41.822843 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:41.823537 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:41.824343 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:41.824362 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.826223 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.826226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:41.827303 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:41.827315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.828107 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.828112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:41.828312 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:41.829236 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:41.829261 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:41.831251 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:41.833841 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:41.833848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:41.833863 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:41.834185 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:41.834625 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:41.834629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:41.834632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:41.834809 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:41.835594 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:41.835601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:41.835898 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 06:15:41.836104 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 06:15:41.836543 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":186,"threads_started":1}
I20260807 06:15:41.837684 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:41.818441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:42161 (local address 127.9.133.190:40181)
0807 06:15:41.818936 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:41.818960 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:41.819683 (+ 723us) server_negotiation.cc:407] Connection header received
0807 06:15:41.821727 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:41.821745 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:41.822276 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:41.822719 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:41.824636 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.826032 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.827440 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.827934 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.828145 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:41.831514 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:41.831520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:41.831533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:41.831571 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:41.833673 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:41.834342 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:41.834349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:41.834353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:41.834502 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:41.834933 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:41.834942 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:41.834944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:41.835445 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:41.835619 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:41.836254 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 06:15:41.836562 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":152,"threads_started":1}
I20260807 06:15:41.839193 590 ts_tablet_manager.cc:1397] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Time spent bootstrapping tablet: real 0.039s user 0.034s sys 0.004s
I20260807 06:15:41.840876 759 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40181
I20260807 06:15:41.841184 759 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:41.841810 759 heartbeater.cc:511] Master 127.9.133.190:40181 requested a full tablet report, sending...
I20260807 06:15:41.842597 590 raft_consensus.cc:348] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:41.843487 590 raft_consensus.cc:729] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b04726cc014061b197d248cbdd2c79, State: Initialized, Role: FOLLOWER
I20260807 06:15:41.844138 590 consensus_queue.cc:260] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } }
I20260807 06:15:41.845978 469 ts_manager.cc:194] Registered new tserver with Master: 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615)
I20260807 06:15:41.848848 590 ts_tablet_manager.cc:1428] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260807 06:15:41.851666 469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:42161
I20260807 06:15:41.855513 759 heartbeater.cc:503] Master 127.9.133.190:40181 was elected leader, sending a full tablet report...
I20260807 06:15:41.862041 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:41.850851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38615 (local address 127.0.0.1:44456)
0807 06:15:41.853363 (+ 2512us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:41.853409 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:41.853620 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:41.856671 (+ 3051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:41.856677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:41.856707 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:41.856864 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:41.856871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.858296 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.858300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:41.859795 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:41.859808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.860912 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.860942 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:41.861139 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:41.861225 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 06:15:41.861446 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 06:15:41.861763 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2259,"thread_start_us":1207,"threads_started":1}
I20260807 06:15:41.861886 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:41.851282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44456 (local address 127.9.133.129:38615)
0807 06:15:41.853047 (+ 1765us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:41.853061 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:41.855970 (+ 2909us) server_negotiation.cc:407] Connection header received
0807 06:15:41.856065 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:41.856071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:41.856301 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:41.856495 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:41.857011 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.858149 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.859948 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:41.860739 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:41.860857 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:41.861023 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:41.861586 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 06:15:41.861702 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1637,"thread_start_us":158,"threads_started":1}
I20260807 06:15:41.865859 655 tablet_service.cc:1514] Processing DeleteTablet for tablet 451918ba4896423a95ddb3c1fa3b019a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44456
I20260807 06:15:41.867204 776 tablet_replica.cc:330] stopping tablet replica
I20260807 06:15:41.867823 776 raft_consensus.cc:2229] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:41.868251 776 raft_consensus.cc:2258] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:41.869784 776 ts_tablet_manager.cc:1905] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:41.875242 776 ts_tablet_manager.cc:1918] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:41.875411 776 log.cc:1196] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal/wals/451918ba4896423a95ddb3c1fa3b019a
I20260807 06:15:41.875803 776 ts_tablet_manager.cc:1939] T 451918ba4896423a95ddb3c1fa3b019a P 55b04726cc014061b197d248cbdd2c79: Deleting consensus metadata
I20260807 06:15:41.878316 449 catalog_manager.cc:4841] TS 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615): tablet 451918ba4896423a95ddb3c1fa3b019a (table test-table [id=e56d3a7bfc834ab281965acc1db17b18]) successfully deleted
W20260807 06:15:41.977887 765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:41.978287 765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:41.978344 765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:41.983276 765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:41.983388 765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:41.983510 765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:15:41.989292 765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:37927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=43491
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:41.990712 765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:41.992084 765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:42.003831 785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:42.003885 791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:42.003888 787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:42.004194 765 server_base.cc:1034] running on GCE node
I20260807 06:15:42.004846 765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:42.005759 765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:42.006970 765 hybrid_clock.cc:648] HybridClock initialized: now 1786083342006945 us; error 40 us; skew 500 ppm
Aug 07 06:15:42 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083342, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:42.010334 765 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:15:42.011554 765 webserver.cc:460] Webserver started at http://127.9.133.130:43491/ using document root <none> and password file <none>
I20260807 06:15:42.012305 765 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:42.012435 765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:42.017377 765 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 06:15:42.019979 803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:42.021528 765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:15:42.021679 765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "9df1009cda874613985fdb2c47731afe"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "ef72359aaef673a211a306a87fcfdd95"
server_key_iv: "1ebc0da8400013a65a65132f07c472b7"
server_key_version: "encryptionkey@0"
I20260807 06:15:42.022389 765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:42.038758 765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:42.042665 765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:42.042891 765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:42.043637 765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:42.044617 765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:42.044695 765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:42.044771 765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:42.044823 765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:42.056115 765 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:37927
I20260807 06:15:42.056136 967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:37927 every 8 connection(s)
I20260807 06:15:42.057435 765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:15:42.063659 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 765
I20260807 06:15:42.064358 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:42227
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:40181
--webserver_port=33115
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 06:15:42 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083342, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:42.077534 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.059927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34625 (local address 127.9.133.190:40181)
0807 06:15:42.060152 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.060157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.061280 (+ 1123us) server_negotiation.cc:407] Connection header received
0807 06:15:42.062951 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.062955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.063132 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.063231 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:42.065533 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.066190 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.067992 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.068448 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.068597 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.072554 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:42.072559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:42.072561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:42.072614 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:42.074516 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.075386 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.075396 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.075397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.075510 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.075973 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.075987 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.075989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.076244 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:42.076372 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.077103 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.077314 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 06:15:42.078883 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.060451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:34625)
0807 06:15:42.061076 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.061132 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:42.062653 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:42.063576 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:42.063588 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:42.064243 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:42.065200 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.065231 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.066365 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.066409 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.067818 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.067833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.068689 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.068692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.069008 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.069858 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:42.069889 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:42.072263 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:42.074709 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:42.074719 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:42.074753 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:42.075249 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:42.075629 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:42.075632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:42.075634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:42.075851 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:42.076375 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:42.076388 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:42.076786 (+ 398us) client_negotiation.cc:769] Sending connection context
0807 06:15:42.077039 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 06:15:42.077547 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":154,"threads_started":1}
I20260807 06:15:42.080688 969 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40181
I20260807 06:15:42.081085 969 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:42.081928 969 heartbeater.cc:511] Master 127.9.133.190:40181 requested a full tablet report, sending...
I20260807 06:15:42.083623 469 ts_manager.cc:194] Registered new tserver with Master: 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
I20260807 06:15:42.084388 469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:34625
W20260807 06:15:42.206920 974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:15:42.207309 974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:15:42.207355 974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:15:42.211580 974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:15:42.211648 974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:15:42.211755 974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:15:42.217381 974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:42227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=33115
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:15:42.218730 974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:15:42.220058 974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:15:42.231616 987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:42.231635 983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:15:42.231624 982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:15:42.231796 974 server_base.cc:1034] running on GCE node
I20260807 06:15:42.232416 974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:15:42.233187 974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:15:42.234416 974 hybrid_clock.cc:648] HybridClock initialized: now 1786083342234398 us; error 29 us; skew 500 ppm
Aug 07 06:15:42 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083342, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:15:42.237978 974 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:15:42.238977 974 webserver.cc:460] Webserver started at http://127.9.133.131:33115/ using document root <none> and password file <none>
I20260807 06:15:42.239689 974 fs_manager.cc:359] Metadata directory not provided
I20260807 06:15:42.239777 974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:15:42.244664 974 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:15:42.247393 997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:42.248987 974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 06:15:42.249146 974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "02523226508346e98a2d555054c618ab"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "5d4195c68f451bb5d0b77e42541eb8fe"
server_key_iv: "45bcd398c017cf386133aa73746e08a4"
server_key_version: "encryptionkey@0"
I20260807 06:15:42.249861 974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:15:42.291448 974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:15:42.295975 974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:15:42.296278 974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:15:42.297133 974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:15:42.298242 974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:15:42.298301 974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:42.298384 974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:15:42.298421 974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:15:42.310958 974 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:42227
I20260807 06:15:42.310984 1178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:42227 every 8 connection(s)
I20260807 06:15:42.312384 974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:15:42.316530 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 974
Aug 07 06:15:42 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083342, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:15:42.326963 1183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.317443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55648 (local address 127.9.133.190:40181)
0807 06:15:42.317762 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.317766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.317917 (+ 151us) server_negotiation.cc:407] Connection header received
0807 06:15:42.318095 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.318098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.318187 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.318292 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:42.319384 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.319985 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.321046 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.321360 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.321443 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.323256 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:42.323259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:42.323261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:42.323282 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:42.324988 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.325454 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.325459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.325461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.325582 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.325902 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.325905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.325906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.326111 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:42.326404 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.326587 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.326730 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":235,"spinlock_wait_cycles":35072,"thread_start_us":117,"threads_started":1}
I20260807 06:15:42.329970 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.314779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:37683 (local address 127.9.133.190:40181)
0807 06:15:42.315089 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.315093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.316095 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 06:15:42.318095 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.318098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.318187 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.318292 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:42.320326 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.320831 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.322541 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.322761 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.322842 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.325746 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:42.325750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:42.325753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:42.325776 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:42.327554 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.328274 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.328282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.328283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.328367 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.328737 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.328740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.328742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.328997 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:42.329097 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.329697 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.329839 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":110,"spinlock_wait_cycles":23552}
I20260807 06:15:42.331166 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.315288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:37683)
0807 06:15:42.315906 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.315960 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:42.317855 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:42.318926 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:42.318940 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:42.319515 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:15:42.320163 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.320187 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.320971 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.320977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.322404 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.322416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.322942 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.322946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.323220 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.323796 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:15:42.323813 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:15:42.325607 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:15:42.327710 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:42.327715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:42.327730 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:42.328160 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:42.328463 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:15:42.328467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:15:42.328469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:15:42.328626 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:15:42.329110 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:15:42.329116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:15:42.329392 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 06:15:42.329610 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 06:15:42.329991 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":147,"threads_started":1}
I20260807 06:15:42.332587 1179 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40181
I20260807 06:15:42.332877 1179 heartbeater.cc:464] Registering TS with master...
I20260807 06:15:42.333526 1179 heartbeater.cc:511] Master 127.9.133.190:40181 requested a full tablet report, sending...
I20260807 06:15:42.334761 469 ts_manager.cc:194] Registered new tserver with Master: 02523226508346e98a2d555054c618ab (127.9.133.131:42227)
I20260807 06:15:42.335505 469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:37683
I20260807 06:15:42.341639 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:15:42.346055 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.342400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.9.133.190:40181)
0807 06:15:42.342548 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.342552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.342767 (+ 215us) server_negotiation.cc:407] Connection header received
0807 06:15:42.342929 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.342932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.343142 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.343245 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:15:42.343503 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.344039 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.344916 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.345132 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.345208 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.345512 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:15:42.345691 (+ 179us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 06:15:42.345927 (+ 236us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55650: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 06:15:42.346119 762 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55650: token expired
W20260807 06:15:42.346560 372 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.133.190:40181)
I20260807 06:15:42.355150 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.347040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55666 (local address 127.9.133.190:40181)
0807 06:15:42.347177 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.347182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.347196 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:15:42.347342 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.347345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.347398 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.347481 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:15:42.348168 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.348683 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.349522 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.349748 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.349855 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.351515 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:15:42.351518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:15:42.351520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:15:42.351568 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:15:42.353080 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.353487 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.353491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.353493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.353572 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:15:42.353842 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:15:42.353853 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:15:42.353855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:15:42.354302 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:15:42.354441 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.354759 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.354876 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 06:15:42.359361 469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:15:42.363706 469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:15:42.373346 655 tablet_service.cc:1467] Processing CreateTablet for tablet 4a34978115324da69406b277a6637152 (DEFAULT_TABLE table=test-table [id=a684b5a3046c41bf8fa861e5c3014b57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:42.374228 655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a34978115324da69406b277a6637152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:42.378127 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.372363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.9.133.130:37927)
0807 06:15:42.372855 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.372867 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.373038 (+ 171us) server_negotiation.cc:407] Connection header received
0807 06:15:42.373170 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.373176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.373387 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.373560 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:42.374066 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.375395 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.376726 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.377419 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.377556 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.377756 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.377817 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.377922 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":233,"threads_started":1}
I20260807 06:15:42.378392 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.371844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:42227 (local address 127.0.0.1:49378)
0807 06:15:42.372185 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.372208 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:42.372320 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:42.373375 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:42.373378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:42.373400 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:42.373569 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.373578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.375528 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.375532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.376750 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.376761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.377972 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.377978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.378083 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.378109 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:15:42.378160 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:15:42.378234 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":47,"thread_start_us":98,"threads_started":1}
I20260807 06:15:42.378260 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.372060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:37927 (local address 127.0.0.1:57710)
0807 06:15:42.372912 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.372929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:42.373049 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:42.373783 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:42.373789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:42.373814 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:42.373907 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.373913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.375542 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.375546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.376576 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.376588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.377567 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.377571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.377674 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.377700 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 06:15:42.377766 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:15:42.377848 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260807 06:15:42.378661 1197 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Bootstrap starting.
I20260807 06:15:42.379155 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.372017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.9.133.131:42227)
0807 06:15:42.372694 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.372710 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.372738 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:15:42.372841 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.372849 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.373160 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.373884 (+ 724us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:42.374092 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.375396 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.376908 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.377820 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.378510 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.378785 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.378845 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.378946 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":209,"threads_started":1}
I20260807 06:15:42.379865 1197 tablet_bootstrap.cc:654] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:42.382556 867 tablet_service.cc:1467] Processing CreateTablet for tablet 4a34978115324da69406b277a6637152 (DEFAULT_TABLE table=test-table [id=a684b5a3046c41bf8fa861e5c3014b57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:42.382556 1069 tablet_service.cc:1467] Processing CreateTablet for tablet 4a34978115324da69406b277a6637152 (DEFAULT_TABLE table=test-table [id=a684b5a3046c41bf8fa861e5c3014b57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:15:42.383951 867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a34978115324da69406b277a6637152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:42.383955 1069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a34978115324da69406b277a6637152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:15:42.384752 1197 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: No bootstrap required, opened a new log
I20260807 06:15:42.384886 1197 ts_tablet_manager.cc:1397] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 06:15:42.385389 1197 raft_consensus.cc:348] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.385535 1197 raft_consensus.cc:374] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:42.385587 1197 raft_consensus.cc:729] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b04726cc014061b197d248cbdd2c79, State: Initialized, Role: FOLLOWER
I20260807 06:15:42.385744 1197 consensus_queue.cc:260] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.386163 1197 ts_tablet_manager.cc:1428] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 06:15:42.393647 1201 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Bootstrap starting.
I20260807 06:15:42.394407 1202 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Bootstrap starting.
I20260807 06:15:42.397285 1202 tablet_bootstrap.cc:654] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:42.397941 1201 tablet_bootstrap.cc:654] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Neither blocks nor log segments found. Creating new log.
I20260807 06:15:42.398891 1201 log.cc:824] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:42.398890 1202 log.cc:824] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Log is configured to *not* fsync() on all Append() calls
I20260807 06:15:42.401202 1201 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: No bootstrap required, opened a new log
I20260807 06:15:42.401238 1202 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: No bootstrap required, opened a new log
I20260807 06:15:42.401467 1201 ts_tablet_manager.cc:1397] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 06:15:42.401468 1202 ts_tablet_manager.cc:1397] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 06:15:42.405153 1201 raft_consensus.cc:348] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.405167 1202 raft_consensus.cc:348] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.405498 1202 raft_consensus.cc:374] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:42.405506 1201 raft_consensus.cc:374] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:15:42.405567 1202 raft_consensus.cc:729] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02523226508346e98a2d555054c618ab, State: Initialized, Role: FOLLOWER
I20260807 06:15:42.405586 1201 raft_consensus.cc:729] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df1009cda874613985fdb2c47731afe, State: Initialized, Role: FOLLOWER
I20260807 06:15:42.406123 1202 consensus_queue.cc:260] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.406124 1201 consensus_queue.cc:260] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.406968 1179 heartbeater.cc:503] Master 127.9.133.190:40181 was elected leader, sending a full tablet report...
I20260807 06:15:42.407222 969 heartbeater.cc:503] Master 127.9.133.190:40181 was elected leader, sending a full tablet report...
I20260807 06:15:42.407384 1202 ts_tablet_manager.cc:1428] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:15:42.407393 1201 ts_tablet_manager.cc:1428] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:15:42.440361 1200 raft_consensus.cc:482] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:15:42.440653 1200 raft_consensus.cc:504] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.442098 1200 leader_election.cc:288] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02523226508346e98a2d555054c618ab (127.9.133.131:42227), 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
I20260807 06:15:42.446590 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.442590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:34261 (local address 127.9.133.130:37927)
0807 06:15:42.442855 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.442861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.442971 (+ 110us) server_negotiation.cc:407] Connection header received
0807 06:15:42.443179 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.443181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.443235 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.443340 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:42.443688 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.444192 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.445444 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.445871 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.446012 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.446082 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.446317 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.446409 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260807 06:15:42.446594 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.442401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:42227 (local address 127.9.133.129:60493)
0807 06:15:42.442783 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.442801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:42.442978 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:42.443306 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:42.443312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:42.443333 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:42.443521 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.443528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.444344 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.444347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.445313 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.445323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.446012 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.446016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.446166 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.446204 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 06:15:42.446297 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 06:15:42.446431 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":180,"thread_start_us":134,"threads_started":1}
I20260807 06:15:42.446590 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.442555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:60493 (local address 127.9.133.131:42227)
0807 06:15:42.442787 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.442792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.442905 (+ 113us) server_negotiation.cc:407] Connection header received
0807 06:15:42.443133 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.443137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.443197 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.443265 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:15:42.443639 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.444199 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.445444 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.445870 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.446017 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.446081 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.446317 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.446408 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 06:15:42.446597 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.442400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:37927 (local address 127.9.133.129:34261)
0807 06:15:42.442803 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.442819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:15:42.443041 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:15:42.443370 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:15:42.443372 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:15:42.443393 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:15:42.443537 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.443542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.444307 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.444312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.445308 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:15:42.445318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.446017 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.446021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:15:42.446170 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.446207 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 06:15:42.446295 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 06:15:42.446422 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":187,"thread_start_us":113,"threads_started":1}
I20260807 06:15:42.447526 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a34978115324da69406b277a6637152" candidate_uuid: "55b04726cc014061b197d248cbdd2c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02523226508346e98a2d555054c618ab" is_pre_election: true
I20260807 06:15:42.447624 893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a34978115324da69406b277a6637152" candidate_uuid: "55b04726cc014061b197d248cbdd2c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9df1009cda874613985fdb2c47731afe" is_pre_election: true
I20260807 06:15:42.447970 893 raft_consensus.cc:2454] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55b04726cc014061b197d248cbdd2c79 in term 0.
I20260807 06:15:42.447973 1105 raft_consensus.cc:2454] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55b04726cc014061b197d248cbdd2c79 in term 0.
I20260807 06:15:42.448545 584 leader_election.cc:302] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 02523226508346e98a2d555054c618ab, 55b04726cc014061b197d248cbdd2c79, 9df1009cda874613985fdb2c47731afe; no voters:
I20260807 06:15:42.448923 1200 raft_consensus.cc:2781] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:15:42.449026 1200 raft_consensus.cc:482] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:15:42.449098 1200 raft_consensus.cc:3037] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:42.450598 1200 raft_consensus.cc:504] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.451134 1200 leader_election.cc:288] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 1 election: Requested vote from peers 02523226508346e98a2d555054c618ab (127.9.133.131:42227), 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
I20260807 06:15:42.451591 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a34978115324da69406b277a6637152" candidate_uuid: "55b04726cc014061b197d248cbdd2c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02523226508346e98a2d555054c618ab"
I20260807 06:15:42.451748 893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a34978115324da69406b277a6637152" candidate_uuid: "55b04726cc014061b197d248cbdd2c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9df1009cda874613985fdb2c47731afe"
I20260807 06:15:42.451776 1105 raft_consensus.cc:3037] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:42.451903 893 raft_consensus.cc:3037] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:15:42.453444 1105 raft_consensus.cc:2454] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55b04726cc014061b197d248cbdd2c79 in term 1.
I20260807 06:15:42.453444 893 raft_consensus.cc:2454] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55b04726cc014061b197d248cbdd2c79 in term 1.
I20260807 06:15:42.453831 584 leader_election.cc:302] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02523226508346e98a2d555054c618ab, 55b04726cc014061b197d248cbdd2c79; no voters:
I20260807 06:15:42.454067 1200 raft_consensus.cc:2781] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:15:42.454367 1200 raft_consensus.cc:686] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 LEADER]: Becoming Leader. State: Replica: 55b04726cc014061b197d248cbdd2c79, State: Running, Role: LEADER
I20260807 06:15:42.454707 1200 consensus_queue.cc:237] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:15:42.458618 469 catalog_manager.cc:5495] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55b04726cc014061b197d248cbdd2c79 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "55b04726cc014061b197d248cbdd2c79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } health_report { overall_health: HEALTHY } } }
I20260807 06:15:42.481182 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.476785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.9.133.129:38615)
0807 06:15:42.477123 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 06:15:42.477131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:15:42.477145 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:15:42.477238 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:15:42.477241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:15:42.477305 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:15:42.477427 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:15:42.477679 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.478252 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.479207 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:15:42.479456 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:15:42.479576 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:15:42.479867 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:15:42.480462 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:15:42.480608 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 06:15:42.480650 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 06:15:42.480719 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":142,"threads_started":1}
I20260807 06:15:42.486095 893 raft_consensus.cc:1261] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Refusing update from remote peer 55b04726cc014061b197d248cbdd2c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:42.486096 1105 raft_consensus.cc:1261] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Refusing update from remote peer 55b04726cc014061b197d248cbdd2c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:15:42.486956 1200 consensus_queue.cc:1035] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:42.487264 1216 consensus_queue.cc:1035] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:15:42.496762 1225 mvcc.cc:205] Tried to move back new op lower bound from 7315797370815188992 to 7315797370697768960. Current Snapshot: MvccSnapshot[applied={T|T < 7315797370815188992}]
I20260807 06:15:42.496762 1226 mvcc.cc:205] Tried to move back new op lower bound from 7315797370815188992 to 7315797370697768960. Current Snapshot: MvccSnapshot[applied={T|T < 7315797370815188992}]
I20260807 06:15:42.506084 469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:42.506350 469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:15:42.509550 469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Sending DeleteTablet for 3 replicas of tablet 4a34978115324da69406b277a6637152
I20260807 06:15:42.510527 1069 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a34978115324da69406b277a6637152 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:42 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:49378
I20260807 06:15:42.510854 867 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a34978115324da69406b277a6637152 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:42 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:57710
I20260807 06:15:42.511343 655 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a34978115324da69406b277a6637152 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:15:42 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:44456
I20260807 06:15:42.511547 1240 tablet_replica.cc:330] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: stopping tablet replica
I20260807 06:15:42.511724 1241 tablet_replica.cc:330] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: stopping tablet replica
I20260807 06:15:42.511842 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 522
I20260807 06:15:42.511890 1241 raft_consensus.cc:2229] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:15:42.515228 1240 raft_consensus.cc:2229] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:42.515717 1240 raft_consensus.cc:2258] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:42.519783 1239 tablet_replica.cc:330] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: stopping tablet replica
I20260807 06:15:42.520253 1239 raft_consensus.cc:2229] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:15:42.520731 1239 raft_consensus.cc:2258] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:15:42.521917 1240 ts_tablet_manager.cc:1905] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:42.522310 1239 ts_tablet_manager.cc:1905] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:15:42.525362 1239 ts_tablet_manager.cc:1918] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:42.525363 1240 ts_tablet_manager.cc:1918] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:15:42.525480 1239 log.cc:1196] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/4a34978115324da69406b277a6637152
I20260807 06:15:42.525480 1240 log.cc:1196] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/4a34978115324da69406b277a6637152
I20260807 06:15:42.525871 1240 ts_tablet_manager.cc:1939] T 4a34978115324da69406b277a6637152 P 9df1009cda874613985fdb2c47731afe: Deleting consensus metadata
I20260807 06:15:42.525871 1239 ts_tablet_manager.cc:1939] T 4a34978115324da69406b277a6637152 P 02523226508346e98a2d555054c618ab: Deleting consensus metadata
I20260807 06:15:42.527627 450 catalog_manager.cc:4841] TS 02523226508346e98a2d555054c618ab (127.9.133.131:42227): tablet 4a34978115324da69406b277a6637152 (table test-table [id=a684b5a3046c41bf8fa861e5c3014b57]) successfully deleted
I20260807 06:15:42.527658 449 catalog_manager.cc:4841] TS 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927): tablet 4a34978115324da69406b277a6637152 (table test-table [id=a684b5a3046c41bf8fa861e5c3014b57]) successfully deleted
W20260807 06:15:42.529181 449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:38615 (error 108)
I20260807 06:15:42.529245 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 765
I20260807 06:15:42.535127 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:15:42.534497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38615 (local address 127.0.0.1:44478)
0807 06:15:42.534715 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 06:15:42.534838 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 06:15:42.535449 449 catalog_manager.cc:4584] TS 55b04726cc014061b197d248cbdd2c79 (127.9.133.129:38615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a34978115324da69406b277a6637152: Network error: Client connection negotiation failed: client connection to 127.9.133.129:38615: connect: Connection refused (error 111)
I20260807 06:15:42.545858 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 974
I20260807 06:15:42.564563 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 424
I20260807 06:16:02.580811 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40181
--webserver_interface=127.9.133.190
--webserver_port=36129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:02.729209 1245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:02.729648 1245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:02.729699 1245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:02.734205 1245 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 06:16:02.734308 1245 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 06:16:02.734328 1245 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 06:16:02.734347 1245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:02.734370 1245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:02.734393 1245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:02.734412 1245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:02.734431 1245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:02.740738 1245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=36129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:02.742511 1245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:02.744025 1245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:16:02.755920 1245 server_base.cc:1034] running on GCE node
W20260807 06:16:02.755779 1254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:02.755770 1259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:02.755779 1252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:02.756959 1245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:02.758539 1245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:02.759733 1245 hybrid_clock.cc:648] HybridClock initialized: now 1786083362759649 us; error 101 us; skew 500 ppm
Aug 07 06:16:02 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083362, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:02.763509 1245 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:02.764724 1245 webserver.cc:460] Webserver started at http://127.9.133.190:36129/ using document root <none> and password file <none>
I20260807 06:16:02.765558 1245 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:02.765671 1245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:02.770877 1245 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:16:02.773892 1269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:02.775627 1245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:02.775805 1245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "ffb1795312064068a6becd649c5b927c"
format_stamp: "Formatted at 2026-08-07 06:15:19 on dist-test-slave-h3ft"
server_key: "7fe2b56b25eb0b0d8626071786043b68"
server_key_iv: "15638947a6b60480f87141a29b348b17"
server_key_version: "encryptionkey@0"
I20260807 06:16:02.776566 1245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:02.799175 1245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:02.803608 1245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:02.803927 1245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:02.813280 1245 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40181
I20260807 06:16:02.813625 1347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40181 every 8 connection(s)
I20260807 06:16:02.814849 1245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:02.820688 1348 sys_catalog.cc:263] Verifying existing consensus state
I20260807 06:16:02.823155 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 1245
I20260807 06:16:02.823144 1348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap starting.
I20260807 06:16:02.823796 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:38615
--local_ip_for_outbound_sockets=127.9.133.129
--tserver_master_addrs=127.9.133.190:40181
--webserver_port=40237
--webserver_interface=127.9.133.129
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:16:02.848472 1348 log.cc:824] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:02.871209 1348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:16:02.871838 1348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: Bootstrap complete.
I20260807 06:16:02.878564 1348 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:16:02.879199 1348 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb1795312064068a6becd649c5b927c, State: Initialized, Role: FOLLOWER
I20260807 06:16:02.879886 1348 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:16:02.880056 1348 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:02.880133 1348 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:02.880245 1348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 2 FOLLOWER]: Advancing to term 3
I20260807 06:16:02.883337 1348 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:16:02.883809 1348 leader_election.cc:302] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffb1795312064068a6becd649c5b927c; no voters:
I20260807 06:16:02.884260 1348 leader_election.cc:288] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 06:16:02.884474 1356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 3 FOLLOWER]: Leader election won for term 3
I20260807 06:16:02.884819 1356 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [term 3 LEADER]: Becoming Leader. State: Replica: ffb1795312064068a6becd649c5b927c, State: Running, Role: LEADER
I20260807 06:16:02.885272 1356 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } }
I20260807 06:16:02.885802 1348 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:02.887501 1357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ffb1795312064068a6becd649c5b927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } } }
I20260807 06:16:02.887648 1357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:02.887456 1359 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffb1795312064068a6becd649c5b927c. Latest consensus state: current_term: 3 leader_uuid: "ffb1795312064068a6becd649c5b927c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb1795312064068a6becd649c5b927c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40181 } } }
I20260807 06:16:02.887910 1359 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:02.891945 1378 catalog_manager.cc:1259] Loaded cluster ID: 32c2ca61d52844efac834b17f38dc473
I20260807 06:16:02.892027 1378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: loading cluster ID for follower catalog manager: success
I20260807 06:16:02.894178 1378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ffb1795312064068a6becd649c5b927c: acquiring CA information for follower catalog manager: success
F20260807 06:16:02.895377 1378 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f456568a9ad google::LogMessage::Fail() at ??:0
@ 0x7f456568d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f456568a4de google::LogMessage::Flush() at ??:0
@ 0x7f456568ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f456834e18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f456ffef989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f456ffcc023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f456ffcc54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f456ffc5652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f456ffc49e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f457000b541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f45678cd512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4568f116db start_thread at ??:0
@ 0x7f456418861f clone at ??:0
W20260807 06:16:03.029670 1351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:03.030269 1351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:03.030364 1351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:03.035836 1351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:03.035964 1351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:03.036077 1351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:03.042994 1351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:38615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=40237
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:03.044838 1351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:03.046334 1351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:03.059674 1419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:03.059655 1417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:03.059664 1416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:03.060655 1351 server_base.cc:1034] running on GCE node
I20260807 06:16:03.061337 1351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:03.062312 1351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:03.063512 1351 hybrid_clock.cc:648] HybridClock initialized: now 1786083363063476 us; error 55 us; skew 500 ppm
Aug 07 06:16:03 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083363, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:03.067651 1351 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:16:03.068959 1351 webserver.cc:460] Webserver started at http://127.9.133.129:40237/ using document root <none> and password file <none>
I20260807 06:16:03.069720 1351 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:03.069793 1351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:03.075273 1351 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:03.078303 1427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.080129 1351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:03.080328 1351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "55b04726cc014061b197d248cbdd2c79"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "182d8693c7a841b47bc6fa7e70fc791d"
server_key_iv: "c343bb48c90a81cc89ff6fc84444bbfa"
server_key_version: "encryptionkey@0"
I20260807 06:16:03.081104 1351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:03.113272 1351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:03.117692 1351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:03.117972 1351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:03.118850 1351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:03.120400 1434 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 06:16:03.127737 1351 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 06:16:03.127879 1351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260807 06:16:03.128000 1351 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 06:16:03.130537 1351 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 06:16:03.130627 1351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260807 06:16:03.131055 1434 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Bootstrap starting.
I20260807 06:16:03.144035 1351 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:38615
I20260807 06:16:03.144354 1580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:38615 every 8 connection(s)
I20260807 06:16:03.145375 1351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:03.146749 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 1351
I20260807 06:16:03.147425 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:37927
--local_ip_for_outbound_sockets=127.9.133.130
--tserver_master_addrs=127.9.133.190:40181
--webserver_port=43491
--webserver_interface=127.9.133.130
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:16:03.157177 1434 log.cc:824] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:03.169953 1434 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 06:16:03.170603 1434 tablet_bootstrap.cc:492] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Bootstrap complete.
I20260807 06:16:03.172219 1434 ts_tablet_manager.cc:1397] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Time spent bootstrapping tablet: real 0.042s user 0.029s sys 0.011s
I20260807 06:16:03.175209 1434 raft_consensus.cc:348] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:16:03.175796 1434 raft_consensus.cc:729] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b04726cc014061b197d248cbdd2c79, State: Initialized, Role: FOLLOWER
I20260807 06:16:03.176397 1434 consensus_queue.cc:260] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:16:03.177579 1434 ts_tablet_manager.cc:1428] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 06:16:03.181670 1589 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:16:03.149260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:40989)
0807 06:16:03.151431 (+ 2171us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.151512 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:03.152785 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:03.180653 (+ 27868us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1862,"thread_start_us":174,"threads_started":1}
W20260807 06:16:03.182395 1582 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:40181 (0 consecutive failures): Network error: Failed to ping master at 127.9.133.190:40181: Client connection negotiation failed: client connection to 127.9.133.190:40181: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 06:16:03.183282 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.182950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:45709)
0807 06:16:03.183108 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.183220 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 06:16:03.184108 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.183895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:36133)
0807 06:16:03.184003 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.184051 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 06:16:03.184348 1582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:16:03.300442 1590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:03.300812 1590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:03.300864 1590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:03.305147 1590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:03.305233 1590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:03.305305 1590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:03.311656 1590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:37927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=43491
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:03.313107 1590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:03.314404 1590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:03.327210 1601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:03.327214 1606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:03.327214 1603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:03.327684 1590 server_base.cc:1034] running on GCE node
I20260807 06:16:03.328367 1590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:03.329324 1590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:03.330642 1590 hybrid_clock.cc:648] HybridClock initialized: now 1786083363330617 us; error 44 us; skew 500 ppm
Aug 07 06:16:03 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083363, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:03.334858 1590 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:16:03.336354 1590 webserver.cc:460] Webserver started at http://127.9.133.130:43491/ using document root <none> and password file <none>
I20260807 06:16:03.337173 1590 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:03.337258 1590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:03.342877 1590 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 06:16:03.346036 1615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.347903 1590 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:03.348089 1590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "9df1009cda874613985fdb2c47731afe"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "ef72359aaef673a211a306a87fcfdd95"
server_key_iv: "1ebc0da8400013a65a65132f07c472b7"
server_key_version: "encryptionkey@0"
I20260807 06:16:03.348867 1590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:03.373180 1590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:03.377398 1590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:03.377686 1590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:03.378567 1590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:03.379731 1590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:03.379822 1590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.379904 1590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:03.379953 1590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.391750 1590 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:37927
I20260807 06:16:03.391811 1787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:37927 every 8 connection(s)
I20260807 06:16:03.393170 1590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:03.398821 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.396149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:52507)
0807 06:16:03.396853 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.397044 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":138,"threads_started":1}
W20260807 06:16:03.399421 1788 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:40181 (0 consecutive failures): Network error: Failed to ping master at 127.9.133.190:40181: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
I20260807 06:16:03.400491 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.400085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:46099)
0807 06:16:03.400245 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.400388 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 06:16:03.400748 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 1590
I20260807 06:16:03.401383 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.401077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:59219)
0807 06:16:03.401219 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.401293 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 06:16:03.401422 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:42227
--local_ip_for_outbound_sockets=127.9.133.131
--tserver_master_addrs=127.9.133.190:40181
--webserver_port=33115
--webserver_interface=127.9.133.131
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:03.401662 1788 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 06:16:03.541909 1792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:03.542331 1792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:03.542387 1792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:03.547065 1792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:03.547173 1792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:03.547242 1792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:03.555884 1792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:42227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=33115
--tserver_master_addrs=127.9.133.190:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:03.557967 1792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:03.559862 1792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:03.573542 1798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:03.573534 1802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:03.573653 1799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:03.574123 1792 server_base.cc:1034] running on GCE node
I20260807 06:16:03.574656 1792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:03.575562 1792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:03.576752 1792 hybrid_clock.cc:648] HybridClock initialized: now 1786083363576732 us; error 29 us; skew 500 ppm
Aug 07 06:16:03 dist-test-slave-h3ft krb5kdc[32042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083363, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:03.580335 1792 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:16:03.581506 1792 webserver.cc:460] Webserver started at http://127.9.133.131:33115/ using document root <none> and password file <none>
I20260807 06:16:03.582250 1792 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:03.582357 1792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:03.587752 1792 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:03.590591 1810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.592284 1792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:03.592459 1792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "02523226508346e98a2d555054c618ab"
format_stamp: "Formatted at 2026-08-07 06:15:20 on dist-test-slave-h3ft"
server_key: "5d4195c68f451bb5d0b77e42541eb8fe"
server_key_iv: "45bcd398c017cf386133aa73746e08a4"
server_key_version: "encryptionkey@0"
I20260807 06:16:03.593297 1792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:03.607300 1792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:03.611642 1792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:03.611908 1792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:03.612730 1792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:03.613916 1792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:03.614001 1792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.614101 1792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:03.614152 1792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:03.625607 1792 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:42227
I20260807 06:16:03.625630 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:42227 every 8 connection(s)
I20260807 06:16:03.627489 1792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:03.632683 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.630366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:53693)
0807 06:16:03.630971 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.631184 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":140,"threads_started":1}
W20260807 06:16:03.633247 1984 heartbeater.cc:644] Failed to heartbeat to 127.9.133.190:40181 (0 consecutive failures): Network error: Failed to ping master at 127.9.133.190:40181: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
I20260807 06:16:03.634024 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.633748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:43331)
0807 06:16:03.633879 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.633943 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 06:16:03.634197 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 1792
I20260807 06:16:03.634717 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:03.634518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:59921)
0807 06:16:03.634624 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:03.634667 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 06:16:03.634943 1984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 06:16:04.185974 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.185236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:59433)
0807 06:16:04.185738 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:04.185819 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":124,"threads_started":1}
I20260807 06:16:04.403077 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.402487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:32853)
0807 06:16:04.402795 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:04.402860 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":100,"threads_started":1}
I20260807 06:16:04.530675 1996 raft_consensus.cc:482] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:04.531090 1996 raft_consensus.cc:504] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:16:04.532775 1996 leader_election.cc:288] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02523226508346e98a2d555054c618ab (127.9.133.131:42227), 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
Aug 07 06:16:04 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083363, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.131@KRBTEST.COM
Aug 07 06:16:04 dist-test-slave-h3ft krb5kdc[32042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083363, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.130@KRBTEST.COM
I20260807 06:16:04.559024 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.533183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:37927 (local address 127.9.133.129:33797)
0807 06:16:04.533688 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:04.533703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:04.533817 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:04.536442 (+ 2625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:04.536446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:04.537008 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:04.537974 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:04.537999 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.540226 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.540230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:04.542097 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:04.542112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.543368 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.543374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:04.543807 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:04.550288 (+ 6481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:04.550341 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:04.553150 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:04.555961 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:04.555965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:04.555967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:04.556291 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:04.556713 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:04.556716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:04.556718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:04.556883 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:04.557656 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:04.557663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:04.557942 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 06:16:04.558159 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 06:16:04.558790 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":623,"spinlock_wait_cycles":70272,"thread_start_us":116,"threads_started":1}
I20260807 06:16:04.559168 1998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.533332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:51181 (local address 127.9.133.131:42227)
0807 06:16:04.533876 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:04.533896 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:04.533959 (+ 63us) server_negotiation.cc:407] Connection header received
0807 06:16:04.535068 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:04.535079 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:04.535685 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:04.536101 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:04.538204 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.539973 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.542290 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.543170 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.543409 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:04.553294 (+ 9885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:04.553302 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:04.553320 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:04.553363 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:04.555730 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:04.556423 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:04.556430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:04.556436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:04.556590 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:04.556978 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:04.556990 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:04.556992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:04.557522 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:04.557678 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:04.558629 (+ 951us) server_negotiation.cc:299] Negotiation successful
0807 06:16:04.558924 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":170,"threads_started":1}
I20260807 06:16:04.559518 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.533407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:33797 (local address 127.9.133.130:37927)
0807 06:16:04.533925 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:04.533949 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:04.533984 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:16:04.535068 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:04.535079 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:04.535685 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:04.536101 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:04.538203 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.539973 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.542290 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.543170 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.543424 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:04.553398 (+ 9974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:04.553403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:04.553423 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:04.553465 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:04.555818 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:04.556437 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:04.556442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:04.556447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:04.556595 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:04.557000 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:04.557004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:04.557005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:04.557522 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:04.557678 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:04.558288 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 06:16:04.559332 (+ 1044us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":187,"threads_started":1}
I20260807 06:16:04.559198 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.533049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:42227 (local address 127.9.133.129:51181)
0807 06:16:04.533364 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:04.533384 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:04.533532 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:04.536217 (+ 2685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:04.536224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:04.537002 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:04.537974 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:04.538000 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.540217 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.540224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:04.542097 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:04.542112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:04.543368 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:04.543374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:04.543808 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:04.550288 (+ 6480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:04.550324 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:04.552927 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:04.555889 (+ 2962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:04.555896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:04.555914 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:04.556285 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:04.556699 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:04.556702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:04.556704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:04.556876 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:04.557655 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:04.557663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:04.557942 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 06:16:04.558160 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 06:16:04.558682 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":531,"spinlock_wait_cycles":22528}
W20260807 06:16:04.562588 1428 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927): Not found: Tablet not found: 4a34978115324da69406b277a6637152
W20260807 06:16:04.563539 1429 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02523226508346e98a2d555054c618ab (127.9.133.131:42227): Not found: Tablet not found: 4a34978115324da69406b277a6637152
I20260807 06:16:04.563633 1429 leader_election.cc:302] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55b04726cc014061b197d248cbdd2c79; no voters: 02523226508346e98a2d555054c618ab, 9df1009cda874613985fdb2c47731afe
I20260807 06:16:04.563987 1996 raft_consensus.cc:2726] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:16:04.636286 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:04.635714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:38235)
0807 06:16:04.636080 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:04.636157 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":96,"threads_started":1}
I20260807 06:16:05.187754 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:05.187181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:52005)
0807 06:16:05.187519 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:05.187605 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":108,"threads_started":1}
I20260807 06:16:05.404712 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:05.404135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:38869)
0807 06:16:05.404472 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:05.404574 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260807 06:16:05.637864 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:05.637239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:37203)
0807 06:16:05.637637 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:05.637705 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":126,"threads_started":1}
I20260807 06:16:06.189306 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:06.188731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:55419)
0807 06:16:06.189077 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:06.189151 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260807 06:16:06.322058 2007 raft_consensus.cc:482] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:06.322444 2007 raft_consensus.cc:504] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:16:06.323238 2007 leader_election.cc:288] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02523226508346e98a2d555054c618ab (127.9.133.131:42227), 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
W20260807 06:16:06.324306 1428 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927): Not found: Tablet not found: 4a34978115324da69406b277a6637152
W20260807 06:16:06.324460 1429 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02523226508346e98a2d555054c618ab (127.9.133.131:42227): Not found: Tablet not found: 4a34978115324da69406b277a6637152
I20260807 06:16:06.324558 1429 leader_election.cc:302] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55b04726cc014061b197d248cbdd2c79; no voters: 02523226508346e98a2d555054c618ab, 9df1009cda874613985fdb2c47731afe
I20260807 06:16:06.324793 2007 raft_consensus.cc:2726] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:16:06.406399 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:06.405758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:36649)
0807 06:16:06.406180 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:06.406258 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":177,"threads_started":1}
I20260807 06:16:06.639415 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:06.638823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:53923)
0807 06:16:06.639211 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:06.639281 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":100,"threads_started":1}
I20260807 06:16:07.191152 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:07.190440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:58819)
0807 06:16:07.190796 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:07.190926 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260807 06:16:07.408324 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:07.407697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:37465)
0807 06:16:07.408072 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:07.408179 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260807 06:16:07.641175 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:07.640553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:36561)
0807 06:16:07.640938 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:07.641025 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260807 06:16:08.192834 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:08.192272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:55791)
0807 06:16:08.192634 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:08.192702 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":100,"threads_started":1}
I20260807 06:16:08.409967 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:08.409410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:37829)
0807 06:16:08.409760 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:08.409827 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":144,"threads_started":1}
I20260807 06:16:08.642683 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:08.642183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:53011)
0807 06:16:08.642492 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:08.642555 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":102,"threads_started":1}
I20260807 06:16:09.194363 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:09.193837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:34447)
0807 06:16:09.194159 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:09.194228 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
I20260807 06:16:09.391577 2024 raft_consensus.cc:482] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:09.391765 2024 raft_consensus.cc:504] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:16:09.392288 2024 leader_election.cc:288] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02523226508346e98a2d555054c618ab (127.9.133.131:42227), 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
W20260807 06:16:09.393278 1428 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927): Not found: Tablet not found: 4a34978115324da69406b277a6637152
W20260807 06:16:09.393428 1429 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02523226508346e98a2d555054c618ab (127.9.133.131:42227): Not found: Tablet not found: 4a34978115324da69406b277a6637152
I20260807 06:16:09.393563 1429 leader_election.cc:302] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55b04726cc014061b197d248cbdd2c79; no voters: 02523226508346e98a2d555054c618ab, 9df1009cda874613985fdb2c47731afe
I20260807 06:16:09.393735 2024 raft_consensus.cc:2726] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:16:09.411494 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:09.410901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:47369)
0807 06:16:09.411294 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:09.411365 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":181,"threads_started":1}
I20260807 06:16:09.644416 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:09.643805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:55955)
0807 06:16:09.644211 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:09.644283 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":113,"threads_started":1}
I20260807 06:16:10.196130 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:10.195506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:46633)
0807 06:16:10.195903 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:10.195979 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260807 06:16:10.413125 2036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:10.412479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:42691)
0807 06:16:10.412926 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:10.412994 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":147,"threads_started":1}
I20260807 06:16:10.646245 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:10.645504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:36427)
0807 06:16:10.645975 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:10.646067 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":98,"threads_started":1}
I20260807 06:16:11.197842 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:11.197216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:33877)
0807 06:16:11.197570 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:11.197702 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260807 06:16:11.414572 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:11.414049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:42721)
0807 06:16:11.414369 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:11.414441 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":99,"threads_started":1}
I20260807 06:16:11.647904 2040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:11.647313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:45799)
0807 06:16:11.647685 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:11.647766 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":98,"threads_started":1}
I20260807 06:16:12.199450 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:12.198857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:59983)
0807 06:16:12.199220 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:12.199297 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":141,"threads_started":1}
I20260807 06:16:12.416116 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:12.415590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:37015)
0807 06:16:12.415903 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:12.415980 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":108,"threads_started":1}
I20260807 06:16:12.649358 2044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:12.648838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:35695)
0807 06:16:12.649159 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:12.649223 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":113,"threads_started":1}
I20260807 06:16:12.859997 2045 raft_consensus.cc:482] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:12.860234 2045 raft_consensus.cc:504] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02523226508346e98a2d555054c618ab" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 42227 } } peers { permanent_uuid: "9df1009cda874613985fdb2c47731afe" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 37927 } } peers { permanent_uuid: "55b04726cc014061b197d248cbdd2c79" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38615 } }
I20260807 06:16:12.860888 2045 leader_election.cc:288] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02523226508346e98a2d555054c618ab (127.9.133.131:42227), 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927)
W20260807 06:16:12.861828 1429 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02523226508346e98a2d555054c618ab (127.9.133.131:42227): Not found: Tablet not found: 4a34978115324da69406b277a6637152
W20260807 06:16:12.861948 1428 leader_election.cc:341] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9df1009cda874613985fdb2c47731afe (127.9.133.130:37927): Not found: Tablet not found: 4a34978115324da69406b277a6637152
I20260807 06:16:12.862092 1428 leader_election.cc:302] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55b04726cc014061b197d248cbdd2c79; no voters: 02523226508346e98a2d555054c618ab, 9df1009cda874613985fdb2c47731afe
I20260807 06:16:12.862272 2045 raft_consensus.cc:2726] T 4a34978115324da69406b277a6637152 P 55b04726cc014061b197d248cbdd2c79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 06:16:13.201107 2047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:13.200541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:58899)
0807 06:16:13.200879 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:13.200945 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":135,"threads_started":1}
I20260807 06:16:13.417629 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:13.417084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:53933)
0807 06:16:13.417432 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:13.417497 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":106,"threads_started":1}
I20260807 06:16:13.650966 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:13.650348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:39357)
0807 06:16:13.650697 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:13.650763 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
I20260807 06:16:14.202749 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:14.202225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:55699)
0807 06:16:14.202543 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:14.202610 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
I20260807 06:16:14.419106 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:14.418541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:34441)
0807 06:16:14.418847 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:14.418912 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260807 06:16:14.652693 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:14.652072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:51415)
0807 06:16:14.652431 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:14.652509 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":125,"threads_started":1}
I20260807 06:16:15.204231 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:15.203689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:45545)
0807 06:16:15.204030 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:15.204096 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":100,"threads_started":1}
I20260807 06:16:15.420699 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:15.420118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:48817)
0807 06:16:15.420489 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:15.420562 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260807 06:16:15.654647 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:15.654044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:49557)
0807 06:16:15.654415 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:15.654492 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":107,"threads_started":1}
I20260807 06:16:16.205729 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:16.205199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:43551)
0807 06:16:16.205532 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:16.205599 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":111,"threads_started":1}
I20260807 06:16:16.422230 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:16.421649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:60325)
0807 06:16:16.422022 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:16.422096 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":126,"threads_started":1}
I20260807 06:16:16.656270 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:16.655649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:52335)
0807 06:16:16.656057 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:16.656127 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":136,"threads_started":1}
I20260807 06:16:17.207355 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:17.206709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:53041)
0807 06:16:17.207141 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:17.207212 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":132,"threads_started":1}
I20260807 06:16:17.424068 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:17.423423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:56379)
0807 06:16:17.423803 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:17.423927 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260807 06:16:17.658063 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:17.657442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:35415)
0807 06:16:17.657806 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:17.657885 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260807 06:16:18.209197 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:18.208658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.129:54597)
0807 06:16:18.209001 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:18.209067 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260807 06:16:18.425897 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:18.425264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.130:33787)
0807 06:16:18.425677 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:18.425753 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":172,"threads_started":1}
I20260807 06:16:18.644037 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 1351
I20260807 06:16:18.660158 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:18.659567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40181 (local address 127.9.133.131:51929)
0807 06:16:18.659922 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:18.660018 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.190:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":131,"threads_started":1}
I20260807 06:16:18.664039 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 1590
I20260807 06:16:18.678139 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 1792
2026-08-07T06:16:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60409 ms)
[----------] 30 tests from SecurityITest (277871 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:18 dist-test-slave-h3ft krb5kdc[2076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:18 dist-test-slave-h3ft krb5kdc[2076](info): set up 2 sockets
Aug 07 06:16:18 dist-test-slave-h3ft krb5kdc[2076](info): commencing operation
krb5kdc: starting...
W20260807 06:16:20.045403 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:20 dist-test-slave-h3ft krb5kdc[2076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:20.202425 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36797
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:36797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:20.336592 2097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:20.337100 2097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:20.337193 2097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:20.341464 2097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:20.341565 2097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:20.341600 2097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:20.341639 2097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:20.341675 2097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:20.347359 2097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:36797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:20.348768 2097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:20.350085 2097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:20.360002 2107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:20.360082 2104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:20.360008 2111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:20.360477 2097 server_base.cc:1034] running on GCE node
I20260807 06:16:20.361231 2097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:20.362476 2097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:20.363641 2097 hybrid_clock.cc:648] HybridClock initialized: now 1786083380363619 us; error 34 us; skew 500 ppm
Aug 07 06:16:20 dist-test-slave-h3ft krb5kdc[2076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:20.366888 2097 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:20.367980 2097 webserver.cc:460] Webserver started at http://127.9.133.190:37717/ using document root <none> and password file <none>
I20260807 06:16:20.368606 2097 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:20.368671 2097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:20.368901 2097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:20.371585 2097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "924a5d96081d5526c9d809a9660593ee"
server_key_iv: "9d250c5696d0c8c4cb8f452808af2748"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.372251 2097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "924a5d96081d5526c9d809a9660593ee"
server_key_iv: "9d250c5696d0c8c4cb8f452808af2748"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.376868 2097 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:20.379835 2123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:20.381412 2097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:20.381572 2097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "924a5d96081d5526c9d809a9660593ee"
server_key_iv: "9d250c5696d0c8c4cb8f452808af2748"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.381697 2097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:20.406944 2097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:20.410809 2097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:20.411082 2097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:20.420003 2097 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:36797
I20260807 06:16:20.420028 2209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:36797 every 8 connection(s)
I20260807 06:16:20.421197 2097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:20.421741 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 2097
I20260807 06:16:20.421931 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:20.422262 9750 external_mini_cluster.cc:1428] Setting key b86077bc22377f0ce3f223834c2fb9c4
I20260807 06:16:20.425079 2210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 06:16:20 dist-test-slave-h3ft krb5kdc[2076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:20.433395 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c: Bootstrap starting.
I20260807 06:16:20.436932 2210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:20.438045 2210 log.cc:824] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:20.439059 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:20.423903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.9.133.190:36797)
0807 06:16:20.424405 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:20.424427 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:20.424479 (+ 52us) server_negotiation.cc:407] Connection header received
0807 06:16:20.425324 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:20.425347 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:20.425749 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:20.426082 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:20.427199 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:20.428297 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:20.429572 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:20.430075 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:20.430196 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:20.433416 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:20.433427 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:20.433446 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:20.433485 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:20.435620 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:20.436078 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:20.436085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:20.436092 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:20.436247 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:20.436517 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:20.436520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:20.436522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:20.437128 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:20.437378 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:20.437595 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 06:16:20.437946 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":161,"threads_started":1}
I20260807 06:16:20.440363 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c: No bootstrap required, opened a new log
I20260807 06:16:20.444278 2210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36797 } }
I20260807 06:16:20.444530 2210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:20.444590 2210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b7723ce18a4a4aaf53eb9a4d963f5c, State: Initialized, Role: FOLLOWER
I20260807 06:16:20.445071 2210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36797 } }
I20260807 06:16:20.445205 2210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:20.445290 2210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:20.445398 2210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:20.446420 2210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36797 } }
I20260807 06:16:20.446828 2210 leader_election.cc:302] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b7723ce18a4a4aaf53eb9a4d963f5c; no voters:
I20260807 06:16:20.447237 2210 leader_election.cc:288] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:20.447355 2219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:20.447669 2219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [term 1 LEADER]: Becoming Leader. State: Replica: 00b7723ce18a4a4aaf53eb9a4d963f5c, State: Running, Role: LEADER
I20260807 06:16:20.447999 2219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36797 } }
I20260807 06:16:20.448647 2210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:20.449916 2224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b7723ce18a4a4aaf53eb9a4d963f5c. Latest consensus state: current_term: 1 leader_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36797 } } }
I20260807 06:16:20.450057 2224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:20.450412 2220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b7723ce18a4a4aaf53eb9a4d963f5c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 36797 } } }
I20260807 06:16:20.450511 2233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:20.450529 2220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:20.454464 2233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:20.461663 2233 catalog_manager.cc:1347] Generated new cluster ID: de4935bb804840f9a7a90617da939043
I20260807 06:16:20.461768 2233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:20.478080 2233 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:20.479372 2233 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:20.489704 2233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c: Generated new TSK 0
I20260807 06:16:20.490777 2233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:16:20.530385 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36797
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:20.666636 2250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:20.667066 2250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:20.667152 2250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:20.671491 2250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:20.671588 2250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:20.671689 2250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:20.677459 2250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:20.678879 2250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:20.680271 2250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:20.692157 2261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:20.692171 2258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:20.692157 2259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:20.692418 2250 server_base.cc:1034] running on GCE node
I20260807 06:16:20.693019 2250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:20.693842 2250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:20.695078 2250 hybrid_clock.cc:648] HybridClock initialized: now 1786083380695060 us; error 31 us; skew 500 ppm
Aug 07 06:16:20 dist-test-slave-h3ft krb5kdc[2076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:20.698410 2250 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:16:20.699633 2250 webserver.cc:460] Webserver started at http://127.9.133.129:33543/ using document root <none> and password file <none>
I20260807 06:16:20.700356 2250 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:20.700429 2250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:20.700682 2250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:20.703328 2250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "a8edfec5af4d407c8a3d5779c56b9d5d"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "c7cffea1ee520e0870d3cabec3e48c39"
server_key_iv: "9e6937554bded6f2e71e0a9391e97af0"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.704020 2250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "a8edfec5af4d407c8a3d5779c56b9d5d"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "c7cffea1ee520e0870d3cabec3e48c39"
server_key_iv: "9e6937554bded6f2e71e0a9391e97af0"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.708756 2250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:20.711911 2269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:20.713469 2250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:16:20.713629 2250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "a8edfec5af4d407c8a3d5779c56b9d5d"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "c7cffea1ee520e0870d3cabec3e48c39"
server_key_iv: "9e6937554bded6f2e71e0a9391e97af0"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.713761 2250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:20.736716 2250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:20.740823 2250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:20.741142 2250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:20.742018 2250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:20.743198 2250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:20.743289 2250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:20.743386 2250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:20.743443 2250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:20.756028 2250 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:44593
I20260807 06:16:20.756052 2445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:44593 every 8 connection(s)
I20260807 06:16:20.757300 2250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:20.760692 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 2250
I20260807 06:16:20.760867 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:16:20.761284 9750 external_mini_cluster.cc:1428] Setting key ede5d48bc47824225af9e094e9cea613
Aug 07 06:16:20 dist-test-slave-h3ft krb5kdc[2076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:16:20.775357 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:20.759751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:47119 (local address 127.9.133.190:36797)
0807 06:16:20.759947 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:20.759953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:20.761151 (+ 1198us) server_negotiation.cc:407] Connection header received
0807 06:16:20.763107 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:20.763110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:20.763182 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:20.763271 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:20.764739 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:20.765649 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:20.767733 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:20.767992 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:20.768081 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:20.771279 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:20.771282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:20.771285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:20.771316 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:20.772938 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:20.773521 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:20.773524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:20.773525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:20.773612 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:20.773986 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:20.773989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:20.773991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:20.774203 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:20.774314 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:20.774989 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 06:16:20.775197 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 06:16:20.776423 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:20.760327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36797 (local address 127.9.133.129:47119)
0807 06:16:20.760930 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:20.761002 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:20.762833 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:20.763487 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:20.763495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:20.763939 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:20.764572 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:20.764588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:20.765834 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:20.765839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:20.767591 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:20.767607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:20.768155 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:20.768158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:20.768389 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:20.769246 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:20.769267 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:20.771071 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:20.773067 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:20.773074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:20.773094 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:20.773400 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:20.773714 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:20.773717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:20.773719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:20.773884 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:20.774314 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:20.774325 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:20.774648 (+ 323us) client_negotiation.cc:769] Sending connection context
0807 06:16:20.774864 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 06:16:20.775261 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":152,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
I20260807 06:16:20.778025 2448 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36797
I20260807 06:16:20.778363 2448 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:20.779172 2448 heartbeater.cc:511] Master 127.9.133.190:36797 requested a full tablet report, sending...
I20260807 06:16:20.781278 2154 ts_manager.cc:194] Registered new tserver with Master: a8edfec5af4d407c8a3d5779c56b9d5d (127.9.133.129:44593)
I20260807 06:16:20.782647 2154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:47119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:16:20.799467 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36797
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:20.939224 2454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:20.939679 2454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:20.939780 2454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:20.944608 2454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:20.944717 2454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:20.944824 2454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:20.951188 2454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:20.952730 2454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:20.954170 2454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:20.966974 2468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:20.967074 2463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:20.966976 2461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:20.967617 2454 server_base.cc:1034] running on GCE node
I20260807 06:16:20.968169 2454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:20.968992 2454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:20.970182 2454 hybrid_clock.cc:648] HybridClock initialized: now 1786083380970159 us; error 39 us; skew 500 ppm
Aug 07 06:16:20 dist-test-slave-h3ft krb5kdc[2076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:20.973706 2454 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:16:20.974877 2454 webserver.cc:460] Webserver started at http://127.9.133.130:44503/ using document root <none> and password file <none>
I20260807 06:16:20.975652 2454 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:20.975749 2454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:20.976032 2454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:20.978701 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "8225f1a7c5ad4ce298fe77f859567353"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "a7911f0c62b24140ac501cd1c21f6341"
server_key_iv: "382510948bd2228e17f58e652c9cb4f9"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.979447 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "8225f1a7c5ad4ce298fe77f859567353"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "a7911f0c62b24140ac501cd1c21f6341"
server_key_iv: "382510948bd2228e17f58e652c9cb4f9"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.984084 2454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:20.987320 2483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:20.988942 2454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:20.989121 2454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "8225f1a7c5ad4ce298fe77f859567353"
format_stamp: "Formatted at 2026-08-07 06:16:20 on dist-test-slave-h3ft"
server_key: "a7911f0c62b24140ac501cd1c21f6341"
server_key_iv: "382510948bd2228e17f58e652c9cb4f9"
server_key_version: "encryptionkey@0"
I20260807 06:16:20.989257 2454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:21.010455 2454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:21.014410 2454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:21.014657 2454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:21.015450 2454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:21.016484 2454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:21.016570 2454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:21.016672 2454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:21.016713 2454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:21.027609 2454 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:38801
I20260807 06:16:21.027631 2659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:38801 every 8 connection(s)
I20260807 06:16:21.028813 2454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:21.029170 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 2454
I20260807 06:16:21.029325 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:16:21.029767 9750 external_mini_cluster.cc:1428] Setting key 8dbb352648986b6a867a36fbe835496b
Aug 07 06:16:21 dist-test-slave-h3ft krb5kdc[2076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083380, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:16:21.047185 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.031477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:39855 (local address 127.9.133.190:36797)
0807 06:16:21.031674 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.031680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.032879 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 06:16:21.034536 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.034542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.034621 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.034705 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:21.036667 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.037249 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.039204 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.039444 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.039551 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.043153 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:21.043156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:21.043159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:21.043188 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:21.044788 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:21.045401 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:21.045404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:21.045406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:21.045490 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:21.045827 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:21.045829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:21.045830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:21.046035 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:21.046135 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.046806 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.046975 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:16:21.048588 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.032070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36797 (local address 127.9.133.130:39855)
0807 06:16:21.032678 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.032741 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.034219 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.034980 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.034994 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.035675 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:21.036509 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.036529 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.037396 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.037402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.039061 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.039078 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.039633 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.039637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.039892 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.040702 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:21.040732 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:21.042918 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:21.044906 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:21.044913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:21.044933 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:21.045300 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:21.045585 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:21.045588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:21.045589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:21.045739 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:21.046155 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:21.046162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:21.046506 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.046742 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.047217 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":167,"threads_started":1}
I20260807 06:16:21.050145 2660 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36797
I20260807 06:16:21.050580 2660 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:21.051394 2660 heartbeater.cc:511] Master 127.9.133.190:36797 requested a full tablet report, sending...
I20260807 06:16:21.052783 2154 ts_manager.cc:194] Registered new tserver with Master: 8225f1a7c5ad4ce298fe77f859567353 (127.9.133.130:38801)
I20260807 06:16:21.053802 2154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:39855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:16:21.067710 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36797
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:21.201179 2667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:21.201532 2667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:21.201617 2667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:21.205832 2667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:21.205932 2667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:21.206029 2667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:21.211840 2667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:21.213337 2667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:21.214728 2667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:21.227028 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:21.227028 2678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:21.227028 2674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:21.227511 2667 server_base.cc:1034] running on GCE node
I20260807 06:16:21.228016 2667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:21.228966 2667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:21.230183 2667 hybrid_clock.cc:648] HybridClock initialized: now 1786083381230143 us; error 50 us; skew 500 ppm
Aug 07 06:16:21 dist-test-slave-h3ft krb5kdc[2076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083381, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:21.233789 2667 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:16:21.234964 2667 webserver.cc:460] Webserver started at http://127.9.133.131:37987/ using document root <none> and password file <none>
I20260807 06:16:21.235726 2667 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:21.235797 2667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:21.236052 2667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:21.238624 2667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "b7583c1a61454bf3ab78c353f48818d8"
format_stamp: "Formatted at 2026-08-07 06:16:21 on dist-test-slave-h3ft"
server_key: "b1bb265492c8a0617102bdacf0761359"
server_key_iv: "b178452d03b2e279f3519d4c1fdeb05d"
server_key_version: "encryptionkey@0"
I20260807 06:16:21.239320 2667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "b7583c1a61454bf3ab78c353f48818d8"
format_stamp: "Formatted at 2026-08-07 06:16:21 on dist-test-slave-h3ft"
server_key: "b1bb265492c8a0617102bdacf0761359"
server_key_iv: "b178452d03b2e279f3519d4c1fdeb05d"
server_key_version: "encryptionkey@0"
I20260807 06:16:21.243868 2667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 06:16:21.246829 2692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:21.248324 2667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:16:21.248456 2667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "b7583c1a61454bf3ab78c353f48818d8"
format_stamp: "Formatted at 2026-08-07 06:16:21 on dist-test-slave-h3ft"
server_key: "b1bb265492c8a0617102bdacf0761359"
server_key_iv: "b178452d03b2e279f3519d4c1fdeb05d"
server_key_version: "encryptionkey@0"
I20260807 06:16:21.248638 2667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:21.262169 2667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:21.266135 2667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:21.266374 2667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:21.267201 2667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:21.268205 2667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:21.268256 2667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:21.268326 2667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:21.268361 2667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:21.279260 2667 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:39271
I20260807 06:16:21.279305 2860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:39271 every 8 connection(s)
I20260807 06:16:21.280457 2667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:21.287714 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 2667
I20260807 06:16:21.287889 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:16:21.288313 9750 external_mini_cluster.cc:1428] Setting key 9b910c7eb8e28a4b5b289786da5c3973
Aug 07 06:16:21 dist-test-slave-h3ft krb5kdc[2076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083381, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:21.296739 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.282758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:55085 (local address 127.9.133.190:36797)
0807 06:16:21.282946 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.282950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.284028 (+ 1078us) server_negotiation.cc:407] Connection header received
0807 06:16:21.285591 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.285595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.285665 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.285773 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:21.287261 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.287916 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.289496 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.289848 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.289971 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.292870 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:21.292874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:21.292877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:21.292912 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:21.294501 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:21.295102 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:21.295105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:21.295107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:21.295201 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:21.295601 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:21.295603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:21.295604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:21.295808 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:21.295898 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.296462 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.296592 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 06:16:21.297848 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.283295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:36797 (local address 127.9.133.131:55085)
0807 06:16:21.283824 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.283874 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.285335 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.285996 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.286003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.286490 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:21.287099 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.287115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.288094 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.288099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.289350 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.289364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.290025 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.290030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.290224 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.290957 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:21.290990 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:21.292691 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:21.294625 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:21.294630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:21.294645 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:21.294924 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:21.295316 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:21.295319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:21.295321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:21.295498 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:21.295908 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:21.295914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:21.296204 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.296386 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.296712 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":139,"threads_started":1}
I20260807 06:16:21.299263 2861 heartbeater.cc:347] Connected to a master server at 127.9.133.190:36797
I20260807 06:16:21.299554 2861 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:21.300204 2861 heartbeater.cc:511] Master 127.9.133.190:36797 requested a full tablet report, sending...
I20260807 06:16:21.301534 2154 ts_manager.cc:194] Registered new tserver with Master: b7583c1a61454bf3ab78c353f48818d8 (127.9.133.131:39271)
I20260807 06:16:21.302330 2154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:55085
I20260807 06:16:21.302985 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:16:21.312983 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.305551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.9.133.190:36797)
0807 06:16:21.305679 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.305683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.305712 (+ 29us) server_negotiation.cc:407] Connection header received
0807 06:16:21.305839 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.305842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.305884 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.305956 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:21.306737 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.307350 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.308240 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.308473 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.308554 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.310030 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:21.310033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:21.310036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:21.310059 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:21.311430 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:21.311828 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:21.311832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:21.311834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:21.311932 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:21.312169 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:21.312171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:21.312172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:21.312418 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:21.312544 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.312741 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.312851 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 06:16:21.316799 2149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:16:21.321071 2149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:16:21.338901 2880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.331996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:44593 (local address 127.0.0.1:46244)
0807 06:16:21.333110 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.333176 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.333330 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.334010 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.334017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.334044 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:21.334235 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.334242 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.336201 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.336204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.337229 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.337240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.338259 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.338263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.338392 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.338448 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.338547 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.338706 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":118,"thread_start_us":125,"threads_started":1}
I20260807 06:16:21.339428 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.332387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:39271 (local address 127.0.0.1:56384)
0807 06:16:21.333621 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.333635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.333716 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.334479 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.334481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.334499 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:21.334619 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.334625 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.336153 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.336158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.337634 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.337646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.338860 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.338863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.338980 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.339092 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.339162 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.339264 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":97,"thread_start_us":128,"threads_started":1}
I20260807 06:16:21.339697 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.332994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.9.133.130:38801)
0807 06:16:21.333860 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.333873 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.333900 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:16:21.334013 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.334018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.334213 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.334398 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:21.334783 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.335888 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.337777 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.338644 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.338787 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.338958 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.339368 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.339498 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":169,"threads_started":1}
I20260807 06:16:21.339715 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.332903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46244 (local address 127.9.133.129:44593)
0807 06:16:21.333281 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.333290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.333318 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:16:21.333518 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.333526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.333793 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.334015 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:21.334698 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.336050 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.337355 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.338114 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.339211 (+ 1097us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.339381 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.339435 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.339532 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":131,"threads_started":1}
I20260807 06:16:21.339794 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.332294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38801 (local address 127.0.0.1:48906)
0807 06:16:21.333399 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.333415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.333551 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.334365 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.334370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.334396 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:21.334550 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.334560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.336035 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.336040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.337638 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.337646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.338795 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.338801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.338926 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.339093 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.339164 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.339264 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":296,"thread_start_us":137,"threads_started":1}
I20260807 06:16:21.340533 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.333476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.9.133.131:39271)
0807 06:16:21.334004 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.334017 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.334041 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:16:21.334124 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.334129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.334319 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.334497 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:21.334708 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.336015 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.337814 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.338727 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.340054 (+ 1327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.340214 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.340265 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.340372 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":195,"threads_started":1}
I20260807 06:16:21.342955 2348 tablet_service.cc:1467] Processing CreateTablet for tablet 2951d7f347b640399f236b0e49fcb24e (DEFAULT_TABLE table=test-table [id=ca8b68a62e9f4264b87bba6137d3b3eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:21.343041 2761 tablet_service.cc:1467] Processing CreateTablet for tablet 2951d7f347b640399f236b0e49fcb24e (DEFAULT_TABLE table=test-table [id=ca8b68a62e9f4264b87bba6137d3b3eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:21.342967 2556 tablet_service.cc:1467] Processing CreateTablet for tablet 2951d7f347b640399f236b0e49fcb24e (DEFAULT_TABLE table=test-table [id=ca8b68a62e9f4264b87bba6137d3b3eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:21.344579 2556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2951d7f347b640399f236b0e49fcb24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:21.344588 2761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2951d7f347b640399f236b0e49fcb24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:21.345206 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2951d7f347b640399f236b0e49fcb24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:21.353348 2885 tablet_bootstrap.cc:492] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: Bootstrap starting.
I20260807 06:16:21.354431 2888 tablet_bootstrap.cc:492] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Bootstrap starting.
I20260807 06:16:21.354431 2887 tablet_bootstrap.cc:492] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Bootstrap starting.
I20260807 06:16:21.356509 2885 tablet_bootstrap.cc:654] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:21.356703 2888 tablet_bootstrap.cc:654] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:21.356703 2887 tablet_bootstrap.cc:654] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:21.357828 2885 log.cc:824] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:21.358194 2888 log.cc:824] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:21.358201 2887 log.cc:824] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:21.360226 2888 tablet_bootstrap.cc:492] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: No bootstrap required, opened a new log
I20260807 06:16:21.360251 2885 tablet_bootstrap.cc:492] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: No bootstrap required, opened a new log
I20260807 06:16:21.360251 2887 tablet_bootstrap.cc:492] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: No bootstrap required, opened a new log
I20260807 06:16:21.360462 2888 ts_tablet_manager.cc:1397] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:16:21.360463 2887 ts_tablet_manager.cc:1397] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:16:21.360463 2885 ts_tablet_manager.cc:1397] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 06:16:21.363822 2888 raft_consensus.cc:348] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.364125 2888 raft_consensus.cc:374] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:21.364183 2888 raft_consensus.cc:729] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7583c1a61454bf3ab78c353f48818d8, State: Initialized, Role: FOLLOWER
I20260807 06:16:21.364686 2887 raft_consensus.cc:348] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.364948 2887 raft_consensus.cc:374] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:21.365002 2887 raft_consensus.cc:729] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8225f1a7c5ad4ce298fe77f859567353, State: Initialized, Role: FOLLOWER
I20260807 06:16:21.364998 2888 consensus_queue.cc:260] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.365351 2885 raft_consensus.cc:348] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.365449 2887 consensus_queue.cc:260] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.365600 2885 raft_consensus.cc:374] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:21.365656 2885 raft_consensus.cc:729] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8edfec5af4d407c8a3d5779c56b9d5d, State: Initialized, Role: FOLLOWER
I20260807 06:16:21.366142 2861 heartbeater.cc:503] Master 127.9.133.190:36797 was elected leader, sending a full tablet report...
I20260807 06:16:21.366154 2885 consensus_queue.cc:260] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.366264 2888 ts_tablet_manager.cc:1428] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 06:16:21.366511 2887 ts_tablet_manager.cc:1428] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 06:16:21.366891 2660 heartbeater.cc:503] Master 127.9.133.190:36797 was elected leader, sending a full tablet report...
I20260807 06:16:21.367147 2885 ts_tablet_manager.cc:1428] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260807 06:16:21.367267 2448 heartbeater.cc:503] Master 127.9.133.190:36797 was elected leader, sending a full tablet report...
I20260807 06:16:21.436777 2895 raft_consensus.cc:482] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:21.437119 2895 raft_consensus.cc:504] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.438669 2895 leader_election.cc:288] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8225f1a7c5ad4ce298fe77f859567353 (127.9.133.130:38801), b7583c1a61454bf3ab78c353f48818d8 (127.9.133.131:39271)
I20260807 06:16:21.443396 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.439193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:54925 (local address 127.9.133.130:38801)
0807 06:16:21.439498 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.439506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.439531 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:16:21.439781 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.439784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.439838 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.439911 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:21.440310 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.440870 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.442214 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.442674 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.442809 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.442878 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.443137 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.443222 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260807 06:16:21.443418 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.439075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:39271 (local address 127.9.133.129:52625)
0807 06:16:21.439507 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.439521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.439638 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.439993 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.439996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.440013 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:21.440175 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.440180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.441044 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.441047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.442055 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.442065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.442838 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.442842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.442964 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.443052 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.443146 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.443216 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":125,"thread_start_us":122,"threads_started":1}
I20260807 06:16:21.443423 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.439516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:52625 (local address 127.9.133.131:39271)
0807 06:16:21.439706 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.439712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.439731 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:16:21.439804 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.439807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.439856 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.439960 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:21.440323 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.440904 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.442193 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.442689 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.442838 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.442902 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.443188 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.443272 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:16:21.443427 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.438965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:38801 (local address 127.9.133.129:54925)
0807 06:16:21.439431 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.439452 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:21.439619 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:21.439957 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:21.439960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:21.439978 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:21.440174 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.440181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.440994 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.441001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.442058 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:21.442068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.442812 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.442816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:21.442942 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.443030 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 06:16:21.443100 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 06:16:21.443208 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":67,"spinlock_wait_cycles":12160,"thread_start_us":196,"threads_started":1}
I20260807 06:16:21.444430 2794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2951d7f347b640399f236b0e49fcb24e" candidate_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7583c1a61454bf3ab78c353f48818d8" is_pre_election: true
I20260807 06:16:21.444463 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2951d7f347b640399f236b0e49fcb24e" candidate_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8225f1a7c5ad4ce298fe77f859567353" is_pre_election: true
I20260807 06:16:21.444806 2588 raft_consensus.cc:2454] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8edfec5af4d407c8a3d5779c56b9d5d in term 0.
I20260807 06:16:21.444821 2794 raft_consensus.cc:2454] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8edfec5af4d407c8a3d5779c56b9d5d in term 0.
I20260807 06:16:21.445391 2270 leader_election.cc:302] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8edfec5af4d407c8a3d5779c56b9d5d, b7583c1a61454bf3ab78c353f48818d8; no voters:
I20260807 06:16:21.445737 2895 raft_consensus.cc:2781] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:16:21.445866 2895 raft_consensus.cc:482] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:16:21.445922 2895 raft_consensus.cc:3037] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:21.447330 2895 raft_consensus.cc:504] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.447749 2895 leader_election.cc:288] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [CANDIDATE]: Term 1 election: Requested vote from peers 8225f1a7c5ad4ce298fe77f859567353 (127.9.133.130:38801), b7583c1a61454bf3ab78c353f48818d8 (127.9.133.131:39271)
I20260807 06:16:21.448096 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2951d7f347b640399f236b0e49fcb24e" candidate_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8225f1a7c5ad4ce298fe77f859567353"
I20260807 06:16:21.448180 2794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2951d7f347b640399f236b0e49fcb24e" candidate_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7583c1a61454bf3ab78c353f48818d8"
I20260807 06:16:21.448256 2588 raft_consensus.cc:3037] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:21.448338 2794 raft_consensus.cc:3037] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:21.449646 2588 raft_consensus.cc:2454] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8edfec5af4d407c8a3d5779c56b9d5d in term 1.
I20260807 06:16:21.449684 2794 raft_consensus.cc:2454] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8edfec5af4d407c8a3d5779c56b9d5d in term 1.
I20260807 06:16:21.450016 2276 leader_election.cc:302] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8225f1a7c5ad4ce298fe77f859567353, a8edfec5af4d407c8a3d5779c56b9d5d; no voters:
I20260807 06:16:21.450280 2895 raft_consensus.cc:2781] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:21.450559 2895 raft_consensus.cc:686] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [term 1 LEADER]: Becoming Leader. State: Replica: a8edfec5af4d407c8a3d5779c56b9d5d, State: Running, Role: LEADER
I20260807 06:16:21.450939 2895 consensus_queue.cc:237] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } }
I20260807 06:16:21.454787 2154 catalog_manager.cc:5495] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d reported cstate change: term changed from 0 to 1, leader changed from <none> to a8edfec5af4d407c8a3d5779c56b9d5d (127.9.133.129). New cstate: current_term: 1 leader_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8edfec5af4d407c8a3d5779c56b9d5d" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 44593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 } health_report { overall_health: UNKNOWN } } }
I20260807 06:16:21.497431 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.493246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.9.133.129:44593)
0807 06:16:21.493435 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.493441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.493458 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:16:21.493656 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.493658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.493730 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.493858 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:16:21.494147 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.494728 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.495717 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.495955 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.496075 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.496433 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:16:21.496998 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:16:21.497144 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.497182 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.497249 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:16:21.505331 2794 raft_consensus.cc:1261] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 1 FOLLOWER]: Refusing update from remote peer a8edfec5af4d407c8a3d5779c56b9d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:21.505331 2588 raft_consensus.cc:1261] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 1 FOLLOWER]: Refusing update from remote peer a8edfec5af4d407c8a3d5779c56b9d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:21.506310 2895 consensus_queue.cc:1035] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7583c1a61454bf3ab78c353f48818d8" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 39271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:21.506559 2899 consensus_queue.cc:1035] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8225f1a7c5ad4ce298fe77f859567353" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 38801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:16:21.508984 2449 tablet.cc:2369] T 2951d7f347b640399f236b0e49fcb24e P a8edfec5af4d407c8a3d5779c56b9d5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:16:21.528867 2147 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:21.529086 2147 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:21.532800 2147 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00b7723ce18a4a4aaf53eb9a4d963f5c: Sending DeleteTablet for 3 replicas of tablet 2951d7f347b640399f236b0e49fcb24e
I20260807 06:16:21.534089 2348 tablet_service.cc:1514] Processing DeleteTablet for tablet 2951d7f347b640399f236b0e49fcb24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:46244
I20260807 06:16:21.534219 2761 tablet_service.cc:1514] Processing DeleteTablet for tablet 2951d7f347b640399f236b0e49fcb24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:56384
I20260807 06:16:21.535130 2920 tablet_replica.cc:330] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: stopping tablet replica
I20260807 06:16:21.535631 2920 raft_consensus.cc:2229] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:21.535761 2556 tablet_service.cc:1514] Processing DeleteTablet for tablet 2951d7f347b640399f236b0e49fcb24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:21 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:48906
I20260807 06:16:21.536096 2920 raft_consensus.cc:2258] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:21.536166 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 2250
I20260807 06:16:21.554561 2922 tablet_replica.cc:330] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: stopping tablet replica
I20260807 06:16:21.555182 2922 raft_consensus.cc:2229] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:21.555716 2922 raft_consensus.cc:2258] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:21.559365 2922 ts_tablet_manager.cc:1905] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:21.559367 2920 ts_tablet_manager.cc:1905] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:21.562760 2920 ts_tablet_manager.cc:1918] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:21.562759 2922 ts_tablet_manager.cc:1918] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:21.562891 2920 log.cc:1196] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/2951d7f347b640399f236b0e49fcb24e
I20260807 06:16:21.562891 2922 log.cc:1196] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/2951d7f347b640399f236b0e49fcb24e
I20260807 06:16:21.563302 2920 ts_tablet_manager.cc:1939] T 2951d7f347b640399f236b0e49fcb24e P b7583c1a61454bf3ab78c353f48818d8: Deleting consensus metadata
I20260807 06:16:21.563557 2922 ts_tablet_manager.cc:1939] T 2951d7f347b640399f236b0e49fcb24e P 8225f1a7c5ad4ce298fe77f859567353: Deleting consensus metadata
I20260807 06:16:21.565044 2128 catalog_manager.cc:4841] TS 8225f1a7c5ad4ce298fe77f859567353 (127.9.133.130:38801): tablet 2951d7f347b640399f236b0e49fcb24e (table test-table [id=ca8b68a62e9f4264b87bba6137d3b3eb]) successfully deleted
I20260807 06:16:21.565423 2124 catalog_manager.cc:4841] TS b7583c1a61454bf3ab78c353f48818d8 (127.9.133.131:39271): tablet 2951d7f347b640399f236b0e49fcb24e (table test-table [id=ca8b68a62e9f4264b87bba6137d3b3eb]) successfully deleted
I20260807 06:16:21.565568 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 2454
W20260807 06:16:21.565582 2125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:44593 (error 108)
I20260807 06:16:21.574456 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.572386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:44593 (local address 127.0.0.1:46262)
0807 06:16:21.574229 (+ 1843us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:21.574331 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:44593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1712}
W20260807 06:16:21.574716 2125 catalog_manager.cc:4584] TS a8edfec5af4d407c8a3d5779c56b9d5d (127.9.133.129:44593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2951d7f347b640399f236b0e49fcb24e: Network error: Client connection negotiation failed: client connection to 127.9.133.129:44593: connect: Connection refused (error 111)
I20260807 06:16:21.578277 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 2667
I20260807 06:16:21.597031 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 2097
2026-08-07T06:16:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2916 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T06:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:21Z Disabled control of system clock
I20260807 06:16:21.639705 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40771
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:16:21.781939 2929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:21.782347 2929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:21.782392 2929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:21.787349 2929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:21.787439 2929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:21.787457 2929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:21.787477 2929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:21.787494 2929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:21.793391 2929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:21.795046 2929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:21.797082 2929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:21.807515 2936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:21.807704 2929 server_base.cc:1034] running on GCE node
W20260807 06:16:21.807583 2939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:21.807530 2937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:21.808707 2929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:21.810160 2929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:21.811352 2929 hybrid_clock.cc:648] HybridClock initialized: now 1786083381811337 us; error 28 us; skew 500 ppm
I20260807 06:16:21.813720 2929 webserver.cc:460] Webserver started at http://127.9.133.190:41789/ using document root <none> and password file <none>
I20260807 06:16:21.814458 2929 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:21.814560 2929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:21.814838 2929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:21.817503 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "80215d11298d446f9abf07749b808d3c"
format_stamp: "Formatted at 2026-08-07 06:16:21 on dist-test-slave-h3ft"
server_key: "4f1431886f5afbee09efb254838a2e30"
server_key_iv: "583132c86bb3bda1d4489bf03af8233a"
server_key_version: "encryptionkey@0"
I20260807 06:16:21.818181 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "80215d11298d446f9abf07749b808d3c"
format_stamp: "Formatted at 2026-08-07 06:16:21 on dist-test-slave-h3ft"
server_key: "4f1431886f5afbee09efb254838a2e30"
server_key_iv: "583132c86bb3bda1d4489bf03af8233a"
server_key_version: "encryptionkey@0"
I20260807 06:16:21.822703 2929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:16:21.825598 2947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:21.827201 2929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:21.827368 2929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "80215d11298d446f9abf07749b808d3c"
format_stamp: "Formatted at 2026-08-07 06:16:21 on dist-test-slave-h3ft"
server_key: "4f1431886f5afbee09efb254838a2e30"
server_key_iv: "583132c86bb3bda1d4489bf03af8233a"
server_key_version: "encryptionkey@0"
I20260807 06:16:21.827517 2929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:21.848033 2929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:21.849293 2929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:21.849571 2929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:21.863752 2929 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40771
I20260807 06:16:21.863785 3024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40771 every 8 connection(s)
I20260807 06:16:21.865717 2929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:21.868505 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 2929
I20260807 06:16:21.868686 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:21.869104 9750 external_mini_cluster.cc:1428] Setting key 653e1ba24570d1c423c5987ea9a0041a
I20260807 06:16:21.871827 3025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:21.882491 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c: Bootstrap starting.
I20260807 06:16:21.886217 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:21.873244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.9.133.190:40771)
0807 06:16:21.874875 (+ 1631us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:21.874894 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:21.874942 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:16:21.876419 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:21.876454 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:21.877259 (+ 805us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:21.877798 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:21.878381 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.880261 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.881551 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:21.882174 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:21.882373 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:21.883753 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:21.883760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:21.883775 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:21.884254 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:21.884322 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:21.884347 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:21.884577 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:21.884703 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 06:16:21.885055 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1138,"thread_start_us":813,"threads_started":1}
I20260807 06:16:21.886932 3025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:21.888283 3025 log.cc:824] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:21.892071 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c: No bootstrap required, opened a new log
I20260807 06:16:21.897687 3025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80215d11298d446f9abf07749b808d3c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40771 } }
I20260807 06:16:21.898018 3025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:21.898073 3025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80215d11298d446f9abf07749b808d3c, State: Initialized, Role: FOLLOWER
I20260807 06:16:21.898633 3025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80215d11298d446f9abf07749b808d3c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40771 } }
I20260807 06:16:21.898790 3025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:21.898849 3025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:21.898937 3025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:21.900202 3025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80215d11298d446f9abf07749b808d3c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40771 } }
I20260807 06:16:21.900600 3025 leader_election.cc:302] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80215d11298d446f9abf07749b808d3c; no voters:
I20260807 06:16:21.901027 3025 leader_election.cc:288] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:21.901211 3034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:21.901461 3034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [term 1 LEADER]: Becoming Leader. State: Replica: 80215d11298d446f9abf07749b808d3c, State: Running, Role: LEADER
I20260807 06:16:21.901824 3034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80215d11298d446f9abf07749b808d3c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40771 } }
I20260807 06:16:21.902689 3025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:21.904065 3042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80215d11298d446f9abf07749b808d3c. Latest consensus state: current_term: 1 leader_uuid: "80215d11298d446f9abf07749b808d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80215d11298d446f9abf07749b808d3c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40771 } } }
I20260807 06:16:21.904072 3041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80215d11298d446f9abf07749b808d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80215d11298d446f9abf07749b808d3c" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40771 } } }
I20260807 06:16:21.904217 3042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:21.904227 3041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:21.904660 3051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:21.908486 3051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:21.915606 3051 catalog_manager.cc:1347] Generated new cluster ID: 6ad74496150c40b392d87bfc3ccd439f
I20260807 06:16:21.915732 3051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:21.923674 3051 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:21.924847 3051 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:21.931375 3051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c: Generated new TSK 0
I20260807 06:16:21.932533 3051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:21.945040 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40771
--builtin_ntp_servers=127.9.133.148:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:16:22.085107 3066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:22.085531 3066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:22.085619 3066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:22.090065 3066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:22.090191 3066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:22.090305 3066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:22.095773 3066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:22.097354 3066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:22.099289 3066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:22.111841 3072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:22.111821 3076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:22.111820 3073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:22.112164 3066 server_base.cc:1034] running on GCE node
I20260807 06:16:22.112802 3066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:22.113651 3066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:22.114833 3066 hybrid_clock.cc:648] HybridClock initialized: now 1786083382114796 us; error 52 us; skew 500 ppm
I20260807 06:16:22.117322 3066 webserver.cc:460] Webserver started at http://127.9.133.129:41615/ using document root <none> and password file <none>
I20260807 06:16:22.118144 3066 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:22.118222 3066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:22.118500 3066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:22.121292 3066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "c23a0bf907414c2ea0d975d5822e874f"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "66aa64cefc670a213b285b172cc21427"
server_key_iv: "a6e74c917076f66e2e23ea0b727a8ed3"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.122087 3066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "c23a0bf907414c2ea0d975d5822e874f"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "66aa64cefc670a213b285b172cc21427"
server_key_iv: "a6e74c917076f66e2e23ea0b727a8ed3"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.127211 3066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 06:16:22.130497 3084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.132542 3066 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:22.132733 3066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "c23a0bf907414c2ea0d975d5822e874f"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "66aa64cefc670a213b285b172cc21427"
server_key_iv: "a6e74c917076f66e2e23ea0b727a8ed3"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.132897 3066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:22.173760 3066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:22.174721 3066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:22.174981 3066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:22.175812 3066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:22.177042 3066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:22.177120 3066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.177220 3066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:22.177263 3066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.187670 3066 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:33563
I20260807 06:16:22.187706 3268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:33563 every 8 connection(s)
I20260807 06:16:22.188804 3066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:22.196385 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 3066
I20260807 06:16:22.196573 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:16:22.196972 9750 external_mini_cluster.cc:1428] Setting key 4c804ee4d64d200b1102713d06e83e0d
I20260807 06:16:22.199692 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40771
--builtin_ntp_servers=127.9.133.148:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:16:22.200315 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.191132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:53569 (local address 127.9.133.190:40771)
0807 06:16:22.191345 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.191351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.192666 (+ 1315us) server_negotiation.cc:407] Connection header received
0807 06:16:22.194269 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.194273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.194338 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.194425 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:22.195544 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.196155 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.197806 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.198129 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.198252 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.199633 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:22.199637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:22.199640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:22.199657 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:22.199691 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:22.199705 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:22.199799 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.200065 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.200147 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:16:22.201336 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.191687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40771 (local address 127.9.133.129:53569)
0807 06:16:22.192461 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.192516 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.194008 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.194633 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.194639 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.194735 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:22.195394 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.195408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.196300 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.196303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.197644 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.197654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.198294 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.198297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.198458 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.199331 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:22.199333 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:22.199382 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:22.199459 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:22.199831 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:22.199842 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:22.199850 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.200053 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.200383 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":127,"threads_started":1}
I20260807 06:16:22.202749 3270 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40771
I20260807 06:16:22.203106 3270 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:22.203918 3270 heartbeater.cc:511] Master 127.9.133.190:40771 requested a full tablet report, sending...
I20260807 06:16:22.205871 2960 ts_manager.cc:194] Registered new tserver with Master: c23a0bf907414c2ea0d975d5822e874f (127.9.133.129:33563)
I20260807 06:16:22.207327 2960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.129:53569
W20260807 06:16:22.336198 3274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:22.336592 3274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:22.336673 3274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:22.341285 3274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:22.341362 3274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:22.341465 3274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:22.346632 3274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:22.348176 3274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:22.350040 3274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:22.362334 3281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:22.362484 3274 server_base.cc:1034] running on GCE node
W20260807 06:16:22.362358 3287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:22.362735 3280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:22.363310 3274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:22.364108 3274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:22.365329 3274 hybrid_clock.cc:648] HybridClock initialized: now 1786083382365305 us; error 36 us; skew 500 ppm
I20260807 06:16:22.367831 3274 webserver.cc:460] Webserver started at http://127.9.133.130:33451/ using document root <none> and password file <none>
I20260807 06:16:22.368607 3274 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:22.368726 3274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:22.369035 3274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:22.371665 3274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "5c06aa8b0b9b68a5583b5f6e508c8f0a"
server_key_iv: "e324adc5cc3e3aca73807e2d60c54184"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.372331 3274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "5c06aa8b0b9b68a5583b5f6e508c8f0a"
server_key_iv: "e324adc5cc3e3aca73807e2d60c54184"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.377033 3274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:22.380203 3294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.381740 3274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:16:22.381909 3274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "5c06aa8b0b9b68a5583b5f6e508c8f0a"
server_key_iv: "e324adc5cc3e3aca73807e2d60c54184"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.382053 3274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:22.394783 3274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:22.395743 3274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:22.395982 3274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:22.396703 3274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:22.397882 3274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:22.397960 3274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.398042 3274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:22.398085 3274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.409448 3274 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:34147
I20260807 06:16:22.409480 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:34147 every 8 connection(s)
I20260807 06:16:22.410619 3274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:22.420324 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 3274
I20260807 06:16:22.420475 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:16:22.420877 9750 external_mini_cluster.cc:1428] Setting key 762c80a121b1428f721175447aa6a520
I20260807 06:16:22.422148 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.413096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:47063 (local address 127.9.133.190:40771)
0807 06:16:22.413302 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.413306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.414622 (+ 1316us) server_negotiation.cc:407] Connection header received
0807 06:16:22.416324 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.416327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.416388 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.416470 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:22.417550 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.418127 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.419540 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.419773 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.419992 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.421285 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:22.421289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:22.421293 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:22.421310 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:22.421339 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:22.421353 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:22.421432 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.421901 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.421980 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 06:16:22.423218 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.413564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40771 (local address 127.9.133.130:47063)
0807 06:16:22.414379 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.414487 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.416085 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.416670 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.416678 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.416782 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:22.417396 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.417412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.418255 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.418258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.419411 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.419422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.419950 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.419954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.420120 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.420975 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:22.420979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:22.421036 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:22.421115 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:22.421480 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:22.421493 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:22.421502 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.421668 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.421983 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":133,"threads_started":1}
I20260807 06:16:22.423448 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40771
--builtin_ntp_servers=127.9.133.148:45977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:16:22.424579 3460 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40771
I20260807 06:16:22.424885 3460 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:22.425549 3460 heartbeater.cc:511] Master 127.9.133.190:40771 requested a full tablet report, sending...
I20260807 06:16:22.426894 2960 ts_manager.cc:194] Registered new tserver with Master: 7fb0e2ac00c2438ea1a0d5780e9d677f (127.9.133.130:34147)
I20260807 06:16:22.427634 2960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.130:47063
W20260807 06:16:22.559365 3466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:22.559691 3466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:22.559736 3466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:22.564076 3466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:22.564147 3466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:22.564226 3466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:22.569412 3466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:45977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:40771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:22.570922 3466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:22.572825 3466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:22.584635 3476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:22.584527 3479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:22.584568 3475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:22.584825 3466 server_base.cc:1034] running on GCE node
I20260807 06:16:22.585460 3466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:22.586261 3466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:22.587448 3466 hybrid_clock.cc:648] HybridClock initialized: now 1786083382587420 us; error 39 us; skew 500 ppm
I20260807 06:16:22.589629 3466 webserver.cc:460] Webserver started at http://127.9.133.131:46177/ using document root <none> and password file <none>
I20260807 06:16:22.590358 3466 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:22.590461 3466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:22.590768 3466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:22.593358 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "76bcea3e948f4b7ca171d28064be5e05"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "544d8c8302687c38ba118f890e475655"
server_key_iv: "d9359765113bc7f15021f4d3074527ee"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.594022 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "76bcea3e948f4b7ca171d28064be5e05"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "544d8c8302687c38ba118f890e475655"
server_key_iv: "d9359765113bc7f15021f4d3074527ee"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.598332 3466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 06:16:22.601279 3486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.602722 3466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 06:16:22.602876 3466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "76bcea3e948f4b7ca171d28064be5e05"
format_stamp: "Formatted at 2026-08-07 06:16:22 on dist-test-slave-h3ft"
server_key: "544d8c8302687c38ba118f890e475655"
server_key_iv: "d9359765113bc7f15021f4d3074527ee"
server_key_version: "encryptionkey@0"
I20260807 06:16:22.603072 3466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:22.626801 3466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:22.627774 3466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:22.627967 3466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:22.628645 3466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:22.629758 3466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:22.629813 3466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.629858 3466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:22.629925 3466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:22.641520 3466 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:45973
I20260807 06:16:22.641551 3655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:45973 every 8 connection(s)
I20260807 06:16:22.642675 3466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:22.643450 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 3466
I20260807 06:16:22.643594 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:16:22.643987 9750 external_mini_cluster.cc:1428] Setting key 7e67a6a928425612903ba5a3246d7c7f
I20260807 06:16:22.654043 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.645205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:59727 (local address 127.9.133.190:40771)
0807 06:16:22.645517 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.645521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.646927 (+ 1406us) server_negotiation.cc:407] Connection header received
0807 06:16:22.648628 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.648631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.648690 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.648763 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:22.649842 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.650450 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.651875 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.652125 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.652213 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.653527 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:22.653530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:22.653533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:22.653549 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:22.653575 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:22.653584 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:22.653650 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.653810 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.653881 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260807 06:16:22.655243 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.645890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:40771 (local address 127.9.133.131:59727)
0807 06:16:22.646746 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.646806 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.648388 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.648943 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.648952 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.649058 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:22.649688 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.649702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.650619 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.650622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.651741 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.651752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.652294 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.652297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.652470 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.653207 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:22.653209 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:22.653261 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:22.653341 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:22.653665 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:22.653673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:22.653677 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.653791 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.654104 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":151,"threads_started":1}
I20260807 06:16:22.656628 3656 heartbeater.cc:347] Connected to a master server at 127.9.133.190:40771
I20260807 06:16:22.656929 3656 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:22.657675 3656 heartbeater.cc:511] Master 127.9.133.190:40771 requested a full tablet report, sending...
I20260807 06:16:22.659091 2960 ts_manager.cc:194] Registered new tserver with Master: 76bcea3e948f4b7ca171d28064be5e05 (127.9.133.131:45973)
I20260807 06:16:22.659763 2960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.131:59727
I20260807 06:16:22.671298 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:16:22.678256 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.673831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.9.133.190:40771)
0807 06:16:22.674002 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.674007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.674224 (+ 217us) server_negotiation.cc:407] Connection header received
0807 06:16:22.674418 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.674426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.674478 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.674569 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:22.674901 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.675500 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.676447 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.676681 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.676811 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.677774 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:22.677778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:22.677780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:22.677793 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:22.677816 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:22.677824 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:22.677910 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.678004 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.678074 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 06:16:22.681793 2960 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:16:22.685709 2960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:16:22.703830 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.697990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.9.133.131:45973)
0807 06:16:22.698859 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.698870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.698898 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:16:22.698989 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.698994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.699214 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.699370 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:22.699649 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.700447 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.702026 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.702905 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.703048 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.703307 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.703466 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.703602 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":216,"threads_started":1}
I20260807 06:16:22.703900 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.696987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33563 (local address 127.0.0.1:50860)
0807 06:16:22.697477 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.697544 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.697705 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.699143 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.699151 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.699179 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:22.699400 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.699413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.700975 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.700978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.702245 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.702257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.703464 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.703468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.703570 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.703594 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.703645 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.703725 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260807 06:16:22.703912 3679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.698000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34147 (local address 127.0.0.1:54816)
0807 06:16:22.698500 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.698517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.698627 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.699453 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.699459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.699483 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:22.699586 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.699591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.701169 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.701172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.702272 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.702279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.703312 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.703315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.703428 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.703449 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.703495 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.703634 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":50,"thread_start_us":127,"threads_started":1}
I20260807 06:16:22.704161 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.698440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.9.133.130:34147)
0807 06:16:22.698905 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.698916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.698939 (+ 23us) server_negotiation.cc:407] Connection header received
0807 06:16:22.699064 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.699072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.699279 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.699744 (+ 465us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:22.699936 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.701051 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.702414 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.703161 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.703587 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.703797 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.703852 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.703972 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":130,"threads_started":1}
I20260807 06:16:22.704463 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.697277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:45973 (local address 127.0.0.1:57220)
0807 06:16:22.698074 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.698089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.698205 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.699373 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.699376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.699395 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:22.699514 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.699523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.700584 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.700587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.701885 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.701895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.703066 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.703070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.703200 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.703272 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.703396 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.704339 (+ 943us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":108,"threads_started":1}
I20260807 06:16:22.705713 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.697158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.9.133.129:33563)
0807 06:16:22.698495 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.698505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.698540 (+ 35us) server_negotiation.cc:407] Connection header received
0807 06:16:22.698642 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.698650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.698908 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.699138 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:22.699561 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.700831 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.702423 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.703244 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.705187 (+ 1943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.705402 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.705457 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.705563 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":127,"threads_started":1}
I20260807 06:16:22.707918 3361 tablet_service.cc:1467] Processing CreateTablet for tablet e0f03424faf44bdd8c8f487b0d62acf4 (DEFAULT_TABLE table=test-table [id=34ca04f0c0894bf3a3b61c99535456cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:22.707922 3564 tablet_service.cc:1467] Processing CreateTablet for tablet e0f03424faf44bdd8c8f487b0d62acf4 (DEFAULT_TABLE table=test-table [id=34ca04f0c0894bf3a3b61c99535456cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:22.708556 3149 tablet_service.cc:1467] Processing CreateTablet for tablet e0f03424faf44bdd8c8f487b0d62acf4 (DEFAULT_TABLE table=test-table [id=34ca04f0c0894bf3a3b61c99535456cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:22.709209 3361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f03424faf44bdd8c8f487b0d62acf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:22.709209 3564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f03424faf44bdd8c8f487b0d62acf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:22.709482 3149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f03424faf44bdd8c8f487b0d62acf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:22.717602 3688 tablet_bootstrap.cc:492] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: Bootstrap starting.
I20260807 06:16:22.718640 3683 tablet_bootstrap.cc:492] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: Bootstrap starting.
I20260807 06:16:22.718640 3687 tablet_bootstrap.cc:492] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Bootstrap starting.
I20260807 06:16:22.720578 3688 tablet_bootstrap.cc:654] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:22.721031 3687 tablet_bootstrap.cc:654] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:22.721036 3683 tablet_bootstrap.cc:654] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:22.721594 3688 log.cc:824] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:22.721966 3687 log.cc:824] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:22.721961 3683 log.cc:824] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:22.724143 3683 tablet_bootstrap.cc:492] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: No bootstrap required, opened a new log
I20260807 06:16:22.724206 3688 tablet_bootstrap.cc:492] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: No bootstrap required, opened a new log
I20260807 06:16:22.724205 3687 tablet_bootstrap.cc:492] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: No bootstrap required, opened a new log
I20260807 06:16:22.724404 3683 ts_tablet_manager.cc:1397] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:16:22.724411 3688 ts_tablet_manager.cc:1397] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:16:22.724403 3687 ts_tablet_manager.cc:1397] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:16:22.728497 3683 raft_consensus.cc:348] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.728807 3683 raft_consensus.cc:374] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:22.728863 3683 raft_consensus.cc:729] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c23a0bf907414c2ea0d975d5822e874f, State: Initialized, Role: FOLLOWER
I20260807 06:16:22.729444 3683 consensus_queue.cc:260] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.729377 3688 raft_consensus.cc:348] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.729368 3687 raft_consensus.cc:348] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.729640 3688 raft_consensus.cc:374] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:22.729640 3687 raft_consensus.cc:374] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:22.729696 3688 raft_consensus.cc:729] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb0e2ac00c2438ea1a0d5780e9d677f, State: Initialized, Role: FOLLOWER
I20260807 06:16:22.729696 3687 raft_consensus.cc:729] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76bcea3e948f4b7ca171d28064be5e05, State: Initialized, Role: FOLLOWER
I20260807 06:16:22.730158 3687 consensus_queue.cc:260] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.730158 3688 consensus_queue.cc:260] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.730793 3683 ts_tablet_manager.cc:1428] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 06:16:22.731298 3687 ts_tablet_manager.cc:1428] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 06:16:22.731310 3270 heartbeater.cc:503] Master 127.9.133.190:40771 was elected leader, sending a full tablet report...
I20260807 06:16:22.731298 3688 ts_tablet_manager.cc:1428] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 06:16:22.731714 3460 heartbeater.cc:503] Master 127.9.133.190:40771 was elected leader, sending a full tablet report...
I20260807 06:16:22.731735 3656 heartbeater.cc:503] Master 127.9.133.190:40771 was elected leader, sending a full tablet report...
I20260807 06:16:22.794657 3696 raft_consensus.cc:482] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:22.795004 3696 raft_consensus.cc:504] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.796634 3696 leader_election.cc:288] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c23a0bf907414c2ea0d975d5822e874f (127.9.133.129:33563), 7fb0e2ac00c2438ea1a0d5780e9d677f (127.9.133.130:34147)
I20260807 06:16:22.801276 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.796938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33563 (local address 127.9.133.131:48677)
0807 06:16:22.797226 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.797247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.797368 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.797735 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.797739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.797761 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:22.797952 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.797964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.798975 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.798979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.799962 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.799972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.800800 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.800804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.800910 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.800939 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.801021 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.801100 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260807 06:16:22.801314 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.797152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:48677 (local address 127.9.133.129:33563)
0807 06:16:22.797327 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.797333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.797365 (+ 32us) server_negotiation.cc:407] Connection header received
0807 06:16:22.797524 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.797531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.797596 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.797694 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:22.798093 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.798878 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.800104 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.800677 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.800781 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.800830 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.801054 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.801144 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 06:16:22.801486 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.796959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34147 (local address 127.9.133.131:33807)
0807 06:16:22.797712 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.797727 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:22.797885 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:22.798206 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:22.798210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:22.798226 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:22.798369 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.798375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.799376 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.799379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.800307 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:22.800317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.801071 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.801074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:22.801172 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.801195 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 06:16:22.801273 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 06:16:22.801341 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":209,"threads_started":1}
I20260807 06:16:22.801553 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.797353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:33807 (local address 127.9.133.130:34147)
0807 06:16:22.797497 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.797502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.797801 (+ 299us) server_negotiation.cc:407] Connection header received
0807 06:16:22.798028 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.798034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.798105 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.798197 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:22.798479 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.799267 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.800431 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.800934 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.801069 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.801114 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.801310 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.801394 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 06:16:22.802188 3389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f03424faf44bdd8c8f487b0d62acf4" candidate_uuid: "76bcea3e948f4b7ca171d28064be5e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" is_pre_election: true
I20260807 06:16:22.802158 3179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f03424faf44bdd8c8f487b0d62acf4" candidate_uuid: "76bcea3e948f4b7ca171d28064be5e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c23a0bf907414c2ea0d975d5822e874f" is_pre_election: true
I20260807 06:16:22.802582 3389 raft_consensus.cc:2454] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bcea3e948f4b7ca171d28064be5e05 in term 0.
I20260807 06:16:22.802567 3179 raft_consensus.cc:2454] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bcea3e948f4b7ca171d28064be5e05 in term 0.
I20260807 06:16:22.803187 3491 leader_election.cc:302] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 76bcea3e948f4b7ca171d28064be5e05, 7fb0e2ac00c2438ea1a0d5780e9d677f, c23a0bf907414c2ea0d975d5822e874f; no voters:
I20260807 06:16:22.803521 3696 raft_consensus.cc:2781] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:16:22.803628 3696 raft_consensus.cc:482] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:16:22.803689 3696 raft_consensus.cc:3037] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:22.805109 3696 raft_consensus.cc:504] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.805565 3696 leader_election.cc:288] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [CANDIDATE]: Term 1 election: Requested vote from peers c23a0bf907414c2ea0d975d5822e874f (127.9.133.129:33563), 7fb0e2ac00c2438ea1a0d5780e9d677f (127.9.133.130:34147)
I20260807 06:16:22.806020 3389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f03424faf44bdd8c8f487b0d62acf4" candidate_uuid: "76bcea3e948f4b7ca171d28064be5e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f"
I20260807 06:16:22.806064 3179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f03424faf44bdd8c8f487b0d62acf4" candidate_uuid: "76bcea3e948f4b7ca171d28064be5e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c23a0bf907414c2ea0d975d5822e874f"
I20260807 06:16:22.806197 3389 raft_consensus.cc:3037] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:22.806210 3179 raft_consensus.cc:3037] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:22.807679 3179 raft_consensus.cc:2454] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bcea3e948f4b7ca171d28064be5e05 in term 1.
I20260807 06:16:22.807683 3389 raft_consensus.cc:2454] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bcea3e948f4b7ca171d28064be5e05 in term 1.
I20260807 06:16:22.808110 3488 leader_election.cc:302] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76bcea3e948f4b7ca171d28064be5e05, 7fb0e2ac00c2438ea1a0d5780e9d677f; no voters:
I20260807 06:16:22.808310 3696 raft_consensus.cc:2781] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:22.808624 3696 raft_consensus.cc:686] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 1 LEADER]: Becoming Leader. State: Replica: 76bcea3e948f4b7ca171d28064be5e05, State: Running, Role: LEADER
I20260807 06:16:22.809051 3696 consensus_queue.cc:237] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } }
I20260807 06:16:22.813139 2960 catalog_manager.cc:5495] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76bcea3e948f4b7ca171d28064be5e05 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "76bcea3e948f4b7ca171d28064be5e05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76bcea3e948f4b7ca171d28064be5e05" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 45973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 } health_report { overall_health: UNKNOWN } } }
I20260807 06:16:22.832806 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.828676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.9.133.131:45973)
0807 06:16:22.828850 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:22.828856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:22.828874 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:16:22.828977 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:22.828980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:22.829051 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:22.829175 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:16:22.829455 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.829999 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.830917 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:22.831224 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:22.831340 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:22.831684 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:16:22.832309 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:16:22.832450 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:22.832500 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 06:16:22.832598 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 06:16:22.840852 3179 raft_consensus.cc:1261] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 1 FOLLOWER]: Refusing update from remote peer 76bcea3e948f4b7ca171d28064be5e05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:22.840852 3389 raft_consensus.cc:1261] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 1 FOLLOWER]: Refusing update from remote peer 76bcea3e948f4b7ca171d28064be5e05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:22.841825 3698 consensus_queue.cc:1035] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb0e2ac00c2438ea1a0d5780e9d677f" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:22.842128 3696 consensus_queue.cc:1035] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c23a0bf907414c2ea0d975d5822e874f" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 33563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:22.853961 3704 mvcc.cc:205] Tried to move back new op lower bound from 7315797536105598976 to 7315797535989108736. Current Snapshot: MvccSnapshot[applied={T|T < 7315797536105598976}]
I20260807 06:16:22.864559 2960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:22.864801 2960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:22.868217 2960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80215d11298d446f9abf07749b808d3c: Sending DeleteTablet for 3 replicas of tablet e0f03424faf44bdd8c8f487b0d62acf4
I20260807 06:16:22.869366 3149 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f03424faf44bdd8c8f487b0d62acf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:22 UTC) from {username='slave'} at 127.0.0.1:50860
I20260807 06:16:22.869588 3361 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f03424faf44bdd8c8f487b0d62acf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:22 UTC) from {username='slave'} at 127.0.0.1:54816
I20260807 06:16:22.870450 3712 tablet_replica.cc:330] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f: stopping tablet replica
I20260807 06:16:22.870550 3564 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f03424faf44bdd8c8f487b0d62acf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:22 UTC) from {username='slave'} at 127.0.0.1:57220
I20260807 06:16:22.870883 3712 raft_consensus.cc:2229] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:22.871840 3712 raft_consensus.cc:2258] T e0f03424faf44bdd8c8f487b0d62acf4 P c23a0bf907414c2ea0d975d5822e874f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:22.871975 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 3066
I20260807 06:16:22.891338 3714 tablet_replica.cc:330] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: stopping tablet replica
I20260807 06:16:22.891342 3716 tablet_replica.cc:330] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: stopping tablet replica
I20260807 06:16:22.891882 3714 raft_consensus.cc:2229] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:22.891880 3716 raft_consensus.cc:2229] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:16:22.892367 3714 raft_consensus.cc:2258] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:22.892450 3716 raft_consensus.cc:2258] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:22.895145 3716 ts_tablet_manager.cc:1905] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:22.896349 3714 ts_tablet_manager.cc:1905] T e0f03424faf44bdd8c8f487b0d62acf4 P 7fb0e2ac00c2438ea1a0d5780e9d677f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 06:16:22.896910 2953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:33563 (error 108)
I20260807 06:16:22.898573 3716 ts_tablet_manager.cc:1918] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:22.898681 3716 log.cc:1196] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/e0f03424faf44bdd8c8f487b0d62acf4
I20260807 06:16:22.899091 3716 ts_tablet_manager.cc:1939] T e0f03424faf44bdd8c8f487b0d62acf4 P 76bcea3e948f4b7ca171d28064be5e05: Deleting consensus metadata
I20260807 06:16:22.899096 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.898644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:33563 (local address 127.0.0.1:50874)
0807 06:16:22.898833 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.898933 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:33563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 06:16:22.899196 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 3274
W20260807 06:16:22.911059 2953 catalog_manager.cc:4584] TS c23a0bf907414c2ea0d975d5822e874f (127.9.133.129:33563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0f03424faf44bdd8c8f487b0d62acf4: Network error: Client connection negotiation failed: client connection to 127.9.133.129:33563: connect: Connection refused (error 111)
I20260807 06:16:22.917781 2952 catalog_manager.cc:4841] TS 76bcea3e948f4b7ca171d28064be5e05 (127.9.133.131:45973): tablet e0f03424faf44bdd8c8f487b0d62acf4 (table test-table [id=34ca04f0c0894bf3a3b61c99535456cf]) successfully deleted
I20260807 06:16:22.924099 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 3466
I20260807 06:16:22.926870 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:22.924419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34147 (local address 127.0.0.1:54824)
0807 06:16:22.926623 (+ 2204us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:22.926711 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.130:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2075}
I20260807 06:16:22.937463 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 2929
2026-08-07T06:16:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1341 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:23 dist-test-slave-h3ft krb5kdc[3722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:23 dist-test-slave-h3ft krb5kdc[3722](info): set up 2 sockets
Aug 07 06:16:23 dist-test-slave-h3ft krb5kdc[3722](info): commencing operation
krb5kdc: starting...
W20260807 06:16:24.230355 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:24 dist-test-slave-h3ft krb5kdc[3722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:24.378759 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35397
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:35397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:24.511556 3744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:24.511936 3744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:24.511984 3744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:24.516321 3744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:24.516407 3744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:24.516427 3744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:24.516445 3744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:24.516460 3744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:24.522416 3744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:35397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:24.524008 3744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:24.525429 3744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:24.536047 3753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:24.536069 3752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:24.536044 3756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:24.536415 3744 server_base.cc:1034] running on GCE node
I20260807 06:16:24.537341 3744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:24.539043 3744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:24.540342 3744 hybrid_clock.cc:648] HybridClock initialized: now 1786083384540306 us; error 50 us; skew 500 ppm
Aug 07 06:16:24 dist-test-slave-h3ft krb5kdc[3722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083384, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:24.550702 3744 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:24.552217 3744 webserver.cc:460] Webserver started at http://127.9.133.190:46297/ using document root <none> and password file <none>
I20260807 06:16:24.553014 3744 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:24.553126 3744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:24.553414 3744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:24.556443 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "e247c34e030e4c8ea2d1e667dc35a4ae"
format_stamp: "Formatted at 2026-08-07 06:16:24 on dist-test-slave-h3ft"
server_key: "436d06fbfbcbb6aa43b3ae0dc3700bbe"
server_key_iv: "520e1c81a78bfc2e8ab9bc4c6157e871"
server_key_version: "encryptionkey@0"
I20260807 06:16:24.557175 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "e247c34e030e4c8ea2d1e667dc35a4ae"
format_stamp: "Formatted at 2026-08-07 06:16:24 on dist-test-slave-h3ft"
server_key: "436d06fbfbcbb6aa43b3ae0dc3700bbe"
server_key_iv: "520e1c81a78bfc2e8ab9bc4c6157e871"
server_key_version: "encryptionkey@0"
I20260807 06:16:24.562496 3744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 06:16:24.565646 3768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:24.567310 3744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:24.567473 3744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "e247c34e030e4c8ea2d1e667dc35a4ae"
format_stamp: "Formatted at 2026-08-07 06:16:24 on dist-test-slave-h3ft"
server_key: "436d06fbfbcbb6aa43b3ae0dc3700bbe"
server_key_iv: "520e1c81a78bfc2e8ab9bc4c6157e871"
server_key_version: "encryptionkey@0"
I20260807 06:16:24.567617 3744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:24.604745 3744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:24.608496 3744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:24.608857 3744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:24.617977 3744 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:35397
I20260807 06:16:24.618031 3844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:35397 every 8 connection(s)
I20260807 06:16:24.619222 3744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:24.622746 3845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:24.628315 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 3744
I20260807 06:16:24.628511 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:24.628851 9750 external_mini_cluster.cc:1428] Setting key 69472cd1d1e19c8069998427e95a2194
I20260807 06:16:24.629989 3845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae: Bootstrap starting.
I20260807 06:16:24.632983 3845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:24.634135 3845 log.cc:824] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:24.636921 3845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae: No bootstrap required, opened a new log
Aug 07 06:16:24 dist-test-slave-h3ft krb5kdc[3722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:24.642239 3845 raft_consensus.cc:348] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35397 } }
I20260807 06:16:24.642589 3845 raft_consensus.cc:374] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:24.642679 3845 raft_consensus.cc:729] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e247c34e030e4c8ea2d1e667dc35a4ae, State: Initialized, Role: FOLLOWER
I20260807 06:16:24.643352 3845 consensus_queue.cc:260] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35397 } }
I20260807 06:16:24.643594 3845 raft_consensus.cc:388] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:24.643687 3845 raft_consensus.cc:482] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:24.643826 3845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:24.645191 3845 raft_consensus.cc:504] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35397 } }
I20260807 06:16:24.645694 3845 leader_election.cc:302] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e247c34e030e4c8ea2d1e667dc35a4ae; no voters:
I20260807 06:16:24.646193 3845 leader_election.cc:288] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:24.646284 3852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:24.646514 3852 raft_consensus.cc:686] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [term 1 LEADER]: Becoming Leader. State: Replica: e247c34e030e4c8ea2d1e667dc35a4ae, State: Running, Role: LEADER
I20260807 06:16:24.646765 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:24.630720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.9.133.190:35397)
0807 06:16:24.631241 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:24.631261 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:24.631306 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:16:24.632088 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:24.632110 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:24.632527 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:24.632879 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:24.634017 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:24.635459 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:24.636793 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:24.637375 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:24.637517 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:24.640742 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:24.640749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:24.640768 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:24.640802 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:24.642913 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:24.643508 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:24.643514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:24.643520 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:24.643731 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:24.644093 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:24.644095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:24.644096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:24.644621 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:24.644802 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:24.645179 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 06:16:24.645577 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":159,"threads_started":1}
I20260807 06:16:24.646880 3852 consensus_queue.cc:237] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35397 } }
I20260807 06:16:24.647855 3845 sys_catalog.cc:564] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:24.648379 3852 sys_catalog.cc:455] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader e247c34e030e4c8ea2d1e667dc35a4ae. Latest consensus state: current_term: 1 leader_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35397 } } }
I20260807 06:16:24.648491 3852 sys_catalog.cc:458] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:24.649288 3854 sys_catalog.cc:455] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e247c34e030e4c8ea2d1e667dc35a4ae" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35397 } } }
I20260807 06:16:24.649410 3854 sys_catalog.cc:458] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:24.649703 3857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:24.653755 3857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:24.660450 3857 catalog_manager.cc:1347] Generated new cluster ID: fcf215d7593a4c85a948fe2167657190
I20260807 06:16:24.660545 3857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:24.702535 3857 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:24.703882 3857 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:24.712651 3857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae: Generated new TSK 0
I20260807 06:16:24.713670 3857 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:16:24.755610 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35397
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:24.896090 3881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:24.896456 3881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:24.896503 3881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:24.900767 3881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:24.900859 3881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:24.900933 3881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:24.906883 3881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:24.908416 3881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:24.909721 3881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:24.922042 3891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:24.922020 3895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:24.922214 3881 server_base.cc:1034] running on GCE node
W20260807 06:16:24.922046 3892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:24.922914 3881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:24.923847 3881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:24.925060 3881 hybrid_clock.cc:648] HybridClock initialized: now 1786083384925033 us; error 42 us; skew 500 ppm
Aug 07 06:16:24 dist-test-slave-h3ft krb5kdc[3722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083384, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:24.928679 3881 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:16:24.929816 3881 webserver.cc:460] Webserver started at http://127.9.133.129:34515/ using document root <none> and password file <none>
I20260807 06:16:24.930586 3881 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:24.930692 3881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:24.930982 3881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:24.933940 3881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "e3bfc05104854c7e81666feb2612c031"
format_stamp: "Formatted at 2026-08-07 06:16:24 on dist-test-slave-h3ft"
server_key: "9b7e0728ee0b2619fb79d9da2163ac18"
server_key_iv: "d725829123bab15049e39d8c3a8303ce"
server_key_version: "encryptionkey@0"
I20260807 06:16:24.934711 3881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "e3bfc05104854c7e81666feb2612c031"
format_stamp: "Formatted at 2026-08-07 06:16:24 on dist-test-slave-h3ft"
server_key: "9b7e0728ee0b2619fb79d9da2163ac18"
server_key_iv: "d725829123bab15049e39d8c3a8303ce"
server_key_version: "encryptionkey@0"
I20260807 06:16:24.940099 3881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 06:16:24.943650 3906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:24.945466 3881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:16:24.945668 3881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "e3bfc05104854c7e81666feb2612c031"
format_stamp: "Formatted at 2026-08-07 06:16:24 on dist-test-slave-h3ft"
server_key: "9b7e0728ee0b2619fb79d9da2163ac18"
server_key_iv: "d725829123bab15049e39d8c3a8303ce"
server_key_version: "encryptionkey@0"
I20260807 06:16:24.945847 3881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:24.978335 3881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:24.982719 3881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:24.983024 3881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:24.983861 3881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:24.984967 3881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:24.985050 3881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:24.985136 3881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:24.985191 3881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:24.996383 3881 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:40791
I20260807 06:16:24.996418 4073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:40791 every 8 connection(s)
I20260807 06:16:24.997596 3881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:25.006125 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 3881
I20260807 06:16:25.006289 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:16:25.006693 9750 external_mini_cluster.cc:1428] Setting key b1542d02c4210c33d153f3f00b498632
Aug 07 06:16:25 dist-test-slave-h3ft krb5kdc[3722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083384, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:25.015565 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.000063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:37193 (local address 127.9.133.190:35397)
0807 06:16:25.000294 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.000298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.001279 (+ 981us) server_negotiation.cc:407] Connection header received
0807 06:16:25.002930 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.002934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.003062 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.003168 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:25.004678 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.005323 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.006735 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.007126 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.007258 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.010530 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:25.010534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:25.010536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:25.010566 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:25.012756 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.013493 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.013497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.013499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.013605 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.014076 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.014078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.014079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.014303 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:25.014380 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.015237 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.015394 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 06:16:25.017190 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.000505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35397 (local address 127.9.133.129:37193)
0807 06:16:25.001080 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.001132 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.002607 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.003376 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.003384 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.003863 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:25.004494 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.004510 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.005468 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.005471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.006595 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.006607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.007320 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.007325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.007598 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.008462 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:25.008487 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:25.010302 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:25.012908 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:25.012915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:25.012935 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:25.013340 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:25.013718 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:25.013724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:25.013729 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:25.013967 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:25.014424 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:25.014435 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:25.014887 (+ 452us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.015183 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.015722 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":159,"threads_started":1}
I20260807 06:16:25.018873 4074 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35397
I20260807 06:16:25.019261 4074 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:25.020097 4074 heartbeater.cc:511] Master 127.9.133.190:35397 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:16:25.022347 3793 ts_manager.cc:194] Registered new tserver with Master: e3bfc05104854c7e81666feb2612c031 (127.9.133.129:40791)
I20260807 06:16:25.023893 3793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:37193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:16:25.043893 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35397
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:25.186707 4084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:25.187112 4084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:25.187170 4084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:25.191408 4084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:25.191483 4084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:25.191547 4084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:25.197558 4084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:25.199122 4084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:25.200472 4084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:25.212029 4093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:25.212047 4091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:25.212049 4090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:25.212481 4084 server_base.cc:1034] running on GCE node
I20260807 06:16:25.212997 4084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:25.213783 4084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:25.215049 4084 hybrid_clock.cc:648] HybridClock initialized: now 1786083385214977 us; error 35 us; skew 500 ppm
Aug 07 06:16:25 dist-test-slave-h3ft krb5kdc[3722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083385, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:25.218431 4084 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:16:25.219527 4084 webserver.cc:460] Webserver started at http://127.9.133.130:39547/ using document root <none> and password file <none>
I20260807 06:16:25.220233 4084 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:25.220348 4084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:25.220623 4084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:25.223297 4084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "243c29743464455aae461de5c58bf5ac"
format_stamp: "Formatted at 2026-08-07 06:16:25 on dist-test-slave-h3ft"
server_key: "9f7c60d84f1938b07f702d352ee86b53"
server_key_iv: "6eb9957c451f55d5219e5e56675e8428"
server_key_version: "encryptionkey@0"
I20260807 06:16:25.224015 4084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "243c29743464455aae461de5c58bf5ac"
format_stamp: "Formatted at 2026-08-07 06:16:25 on dist-test-slave-h3ft"
server_key: "9f7c60d84f1938b07f702d352ee86b53"
server_key_iv: "6eb9957c451f55d5219e5e56675e8428"
server_key_version: "encryptionkey@0"
I20260807 06:16:25.228636 4084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:25.231421 4104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:25.232923 4084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:16:25.233068 4084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "243c29743464455aae461de5c58bf5ac"
format_stamp: "Formatted at 2026-08-07 06:16:25 on dist-test-slave-h3ft"
server_key: "9f7c60d84f1938b07f702d352ee86b53"
server_key_iv: "6eb9957c451f55d5219e5e56675e8428"
server_key_version: "encryptionkey@0"
I20260807 06:16:25.233191 4084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:25.262897 4084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:25.266353 4084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:25.266603 4084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:25.267382 4084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:25.268388 4084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:25.268461 4084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:25.268563 4084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:25.268610 4084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:25.279137 4084 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:34693
I20260807 06:16:25.279181 4267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:34693 every 8 connection(s)
I20260807 06:16:25.280236 4084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:25.284003 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 4084
I20260807 06:16:25.284185 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:16:25.284652 9750 external_mini_cluster.cc:1428] Setting key b5564af26533129a555a071f04c24179
Aug 07 06:16:25 dist-test-slave-h3ft krb5kdc[3722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083385, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:25.297549 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.282486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:33593 (local address 127.9.133.190:35397)
0807 06:16:25.282687 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.282692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.283694 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 06:16:25.285678 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.285682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.285738 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.285823 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:25.287635 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.288237 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.289858 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.290112 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.290230 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.293124 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:25.293129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:25.293132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:25.293173 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:25.295112 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.295656 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.295660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.295662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.295777 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.296162 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.296169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.296171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.296447 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:25.296569 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.297123 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.297307 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:16:25.298445 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.282958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35397 (local address 127.9.133.130:33593)
0807 06:16:25.283511 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.283562 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.285423 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.286035 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.286042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.286624 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:25.287464 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.287489 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.288371 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.288377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.289725 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.289736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.290279 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.290284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.290479 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.291238 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:25.291256 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:25.292927 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:25.295227 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:25.295232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:25.295246 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:25.295538 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:25.295884 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:25.295886 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:25.295888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:25.296045 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:25.296566 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:25.296572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:25.296842 (+ 270us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.297010 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.297314 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":133,"threads_started":1}
I20260807 06:16:25.300140 4268 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35397
I20260807 06:16:25.300426 4268 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:16:25.301087 4268 heartbeater.cc:511] Master 127.9.133.190:35397 requested a full tablet report, sending...
I20260807 06:16:25.302537 3793 ts_manager.cc:194] Registered new tserver with Master: 243c29743464455aae461de5c58bf5ac (127.9.133.130:34693)
I20260807 06:16:25.303512 3793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:33593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:16:25.319705 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35397
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:25.452479 4276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:25.452847 4276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:25.452934 4276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:25.457131 4276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:25.457250 4276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:25.457360 4276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:25.462949 4276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:25.464341 4276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:25.465693 4276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:25.477830 4285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:25.477825 4283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:25.477823 4282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:25.478163 4276 server_base.cc:1034] running on GCE node
I20260807 06:16:25.478672 4276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:25.479530 4276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:25.480728 4276 hybrid_clock.cc:648] HybridClock initialized: now 1786083385480682 us; error 65 us; skew 500 ppm
Aug 07 06:16:25 dist-test-slave-h3ft krb5kdc[3722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083385, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:25.484012 4276 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:16:25.485045 4276 webserver.cc:460] Webserver started at http://127.9.133.131:37461/ using document root <none> and password file <none>
I20260807 06:16:25.485697 4276 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:25.485761 4276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:25.485970 4276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:25.488512 4276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "fc711a6bf2254e198f0fee7228c84c13"
format_stamp: "Formatted at 2026-08-07 06:16:25 on dist-test-slave-h3ft"
server_key: "f5e17c3814389041cb16ffdd647993d8"
server_key_iv: "53264485c89daf863612b34b48760c4f"
server_key_version: "encryptionkey@0"
I20260807 06:16:25.489136 4276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "fc711a6bf2254e198f0fee7228c84c13"
format_stamp: "Formatted at 2026-08-07 06:16:25 on dist-test-slave-h3ft"
server_key: "f5e17c3814389041cb16ffdd647993d8"
server_key_iv: "53264485c89daf863612b34b48760c4f"
server_key_version: "encryptionkey@0"
I20260807 06:16:25.493528 4276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:25.496400 4297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:25.497841 4276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:16:25.497987 4276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "fc711a6bf2254e198f0fee7228c84c13"
format_stamp: "Formatted at 2026-08-07 06:16:25 on dist-test-slave-h3ft"
server_key: "f5e17c3814389041cb16ffdd647993d8"
server_key_iv: "53264485c89daf863612b34b48760c4f"
server_key_version: "encryptionkey@0"
I20260807 06:16:25.498103 4276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:25.542301 4276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:25.546325 4276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:25.546605 4276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:25.547618 4276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:25.548853 4276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:25.548918 4276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:25.549023 4276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:25.549065 4276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:25.561662 4276 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:43391
I20260807 06:16:25.561720 4461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:43391 every 8 connection(s)
I20260807 06:16:25.562839 4276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:25.569453 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 4276
I20260807 06:16:25.569638 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:16:25.570044 9750 external_mini_cluster.cc:1428] Setting key dfcb56123e12ba6be13cd5f74e53b9f2
Aug 07 06:16:25 dist-test-slave-h3ft krb5kdc[3722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083385, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:25.579825 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.565184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:36395 (local address 127.9.133.190:35397)
0807 06:16:25.565382 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.565388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.566399 (+ 1011us) server_negotiation.cc:407] Connection header received
0807 06:16:25.568012 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.568016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.568076 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.568159 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:25.569658 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.570499 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.572275 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.572519 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.572651 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.575764 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:25.575768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:25.575770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:25.575807 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:25.577320 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.577921 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.577925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.577931 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.578021 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.578427 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.578430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.578432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.578656 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:25.578772 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.579470 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.579625 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 06:16:25.580928 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.565640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35397 (local address 127.9.133.131:36395)
0807 06:16:25.566199 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.566248 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.567763 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.568354 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.568361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.568799 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:25.569438 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.569453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.570656 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.570661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.572126 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.572137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.572697 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.572702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.572974 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.573714 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:25.573732 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:25.575556 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:25.577459 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:25.577466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:25.577487 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:25.577796 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:25.578120 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:25.578124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:25.578126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:25.578315 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:25.578772 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:25.578781 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:25.579192 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.579411 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.579838 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":134,"threads_started":1}
I20260807 06:16:25.582304 4462 heartbeater.cc:347] Connected to a master server at 127.9.133.190:35397
I20260807 06:16:25.582638 4462 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:25.583328 4462 heartbeater.cc:511] Master 127.9.133.190:35397 requested a full tablet report, sending...
I20260807 06:16:25.584751 3793 ts_manager.cc:194] Registered new tserver with Master: fc711a6bf2254e198f0fee7228c84c13 (127.9.133.131:43391)
I20260807 06:16:25.585436 3793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:36395
I20260807 06:16:25.597270 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:16:25.608516 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.599795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.9.133.190:35397)
0807 06:16:25.599948 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.599952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.600152 (+ 200us) server_negotiation.cc:407] Connection header received
0807 06:16:25.600272 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.600275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.600324 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.600404 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:25.601196 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.601784 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.602688 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.603034 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.603119 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.604903 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:25.604906 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:25.604909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:25.604936 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:25.606535 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.607068 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.607071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.607072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.607154 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:25.607560 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:25.607562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:25.607563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:25.607799 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:25.607927 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.608233 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.608328 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 06:16:25.612632 3793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:16:25.616482 3793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:16:25.635067 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.626991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:34693 (local address 127.0.0.1:35560)
0807 06:16:25.627931 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.627989 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.628151 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.629673 (+ 1522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.629676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.629711 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:25.629945 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.629956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.631640 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.631644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.633275 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.633286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.634462 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.634465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.634591 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.634636 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.634730 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.634854 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":66,"spinlock_wait_cycles":2560,"thread_start_us":98,"threads_started":1}
I20260807 06:16:25.635476 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.627935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.9.133.129:40791)
0807 06:16:25.628345 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.628354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.628775 (+ 421us) server_negotiation.cc:407] Connection header received
0807 06:16:25.629015 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.629022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.629280 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.629433 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:25.630126 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.631618 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.633489 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.634421 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.634509 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.634691 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.635142 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.635273 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":154,"threads_started":1}
I20260807 06:16:25.635667 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.627395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.9.133.130:34693)
0807 06:16:25.629141 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.629157 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.629185 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:16:25.629275 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.629281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.629472 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.630125 (+ 653us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:25.630341 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.631485 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.633436 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.634335 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.635166 (+ 831us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.635322 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.635374 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.635471 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1620,"thread_start_us":203,"threads_started":1}
I20260807 06:16:25.635987 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.628376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:40791 (local address 127.0.0.1:33800)
0807 06:16:25.628660 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.628675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.628854 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.629659 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.629666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.629709 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:25.629944 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.629953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.631772 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.631776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.633319 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.633330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.634795 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.634799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.634905 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.634950 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.635721 (+ 771us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.635823 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":57,"spinlock_wait_cycles":20224,"thread_start_us":120,"threads_started":1}
I20260807 06:16:25.637149 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.628912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:43391 (local address 127.0.0.1:56764)
0807 06:16:25.631171 (+ 2259us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.631185 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.631352 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.632077 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.632080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.632100 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:25.632182 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.632187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.634435 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.634439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.635527 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.635541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.636731 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.636734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.636835 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.636880 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.636953 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.637030 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2194,"thread_start_us":109,"threads_started":1}
I20260807 06:16:25.637290 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.629402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.9.133.131:43391)
0807 06:16:25.629938 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.629951 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.631283 (+ 1332us) server_negotiation.cc:407] Connection header received
0807 06:16:25.631493 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.631500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.631715 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.631846 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:25.632315 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.634262 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.635669 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.636453 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.636520 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.636623 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.637000 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.637118 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":190,"threads_started":1}
I20260807 06:16:25.639533 4172 tablet_service.cc:1467] Processing CreateTablet for tablet bbbf88d005c54ed384ba086c83187035 (DEFAULT_TABLE table=test-table [id=8a0783cb8a644a5282932ad646e81157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:25.639521 3979 tablet_service.cc:1467] Processing CreateTablet for tablet bbbf88d005c54ed384ba086c83187035 (DEFAULT_TABLE table=test-table [id=8a0783cb8a644a5282932ad646e81157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:25.640877 4172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbf88d005c54ed384ba086c83187035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:25.640883 3979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbf88d005c54ed384ba086c83187035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:25.641448 4368 tablet_service.cc:1467] Processing CreateTablet for tablet bbbf88d005c54ed384ba086c83187035 (DEFAULT_TABLE table=test-table [id=8a0783cb8a644a5282932ad646e81157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:25.642737 4368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbf88d005c54ed384ba086c83187035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:25.649333 4484 tablet_bootstrap.cc:492] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: Bootstrap starting.
I20260807 06:16:25.651219 4486 tablet_bootstrap.cc:492] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Bootstrap starting.
I20260807 06:16:25.651252 4485 tablet_bootstrap.cc:492] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Bootstrap starting.
I20260807 06:16:25.652530 4484 tablet_bootstrap.cc:654] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:25.653478 4486 tablet_bootstrap.cc:654] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:25.653748 4484 log.cc:824] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:25.654343 4485 tablet_bootstrap.cc:654] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:25.654870 4486 log.cc:824] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:25.655304 4485 log.cc:824] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:25.656844 4486 tablet_bootstrap.cc:492] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: No bootstrap required, opened a new log
I20260807 06:16:25.656878 4484 tablet_bootstrap.cc:492] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: No bootstrap required, opened a new log
I20260807 06:16:25.657089 4486 ts_tablet_manager.cc:1397] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:16:25.657094 4484 ts_tablet_manager.cc:1397] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 06:16:25.657548 4485 tablet_bootstrap.cc:492] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: No bootstrap required, opened a new log
I20260807 06:16:25.657756 4485 ts_tablet_manager.cc:1397] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:16:25.660600 4486 raft_consensus.cc:348] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.660931 4486 raft_consensus.cc:374] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:25.660987 4486 raft_consensus.cc:729] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc711a6bf2254e198f0fee7228c84c13, State: Initialized, Role: FOLLOWER
I20260807 06:16:25.661559 4486 consensus_queue.cc:260] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.662313 4484 raft_consensus.cc:348] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.662312 4485 raft_consensus.cc:348] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.662601 4484 raft_consensus.cc:374] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:25.662600 4485 raft_consensus.cc:374] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:25.662667 4484 raft_consensus.cc:729] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bfc05104854c7e81666feb2612c031, State: Initialized, Role: FOLLOWER
I20260807 06:16:25.662667 4485 raft_consensus.cc:729] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243c29743464455aae461de5c58bf5ac, State: Initialized, Role: FOLLOWER
I20260807 06:16:25.662793 4486 ts_tablet_manager.cc:1428] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 06:16:25.663177 4484 consensus_queue.cc:260] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.663177 4485 consensus_queue.cc:260] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.663342 4462 heartbeater.cc:503] Master 127.9.133.190:35397 was elected leader, sending a full tablet report...
I20260807 06:16:25.664238 4484 ts_tablet_manager.cc:1428] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: Time spent starting tablet: real 0.007s user 0.003s sys 0.005s
I20260807 06:16:25.664238 4485 ts_tablet_manager.cc:1428] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:16:25.664616 4074 heartbeater.cc:503] Master 127.9.133.190:35397 was elected leader, sending a full tablet report...
I20260807 06:16:25.664808 4268 heartbeater.cc:503] Master 127.9.133.190:35397 was elected leader, sending a full tablet report...
I20260807 06:16:25.697109 4493 raft_consensus.cc:482] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:25.697466 4493 raft_consensus.cc:504] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.699363 4493 leader_election.cc:288] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3bfc05104854c7e81666feb2612c031 (127.9.133.129:40791), fc711a6bf2254e198f0fee7228c84c13 (127.9.133.131:43391)
I20260807 06:16:25.703450 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.699632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:40791 (local address 127.9.133.130:38795)
0807 06:16:25.699786 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.699810 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.699935 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.700299 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.700302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.700324 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:25.700462 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.700468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.701260 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.701265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.702195 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.702204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.702937 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.702941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.703117 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.703149 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.703239 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.703308 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":23}
I20260807 06:16:25.703534 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.699809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:38795 (local address 127.9.133.129:40791)
0807 06:16:25.700001 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.700006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.700026 (+ 20us) server_negotiation.cc:407] Connection header received
0807 06:16:25.700101 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.700105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.700170 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.700281 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:25.700588 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.701142 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.702323 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.702795 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.702961 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.703057 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.703265 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.703369 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:16:25.704404 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.699639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:43391 (local address 127.9.133.130:43637)
0807 06:16:25.700251 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.700263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:25.700406 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:25.700710 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:25.700714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:25.700732 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:25.700876 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.700883 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.701717 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.701724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.702771 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:25.702781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.703876 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.703881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:25.703994 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.704026 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 06:16:25.704123 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 06:16:25.704221 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":145,"thread_start_us":189,"threads_started":1}
I20260807 06:16:25.704420 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.699936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:43637 (local address 127.9.133.131:43391)
0807 06:16:25.700109 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.700114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.700362 (+ 248us) server_negotiation.cc:407] Connection header received
0807 06:16:25.700549 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.700552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.700603 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.700699 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:25.701003 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.701598 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.702922 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.703712 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.703891 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.703957 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.704178 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.704274 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 06:16:25.704850 4008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf88d005c54ed384ba086c83187035" candidate_uuid: "243c29743464455aae461de5c58bf5ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bfc05104854c7e81666feb2612c031" is_pre_election: true
I20260807 06:16:25.705152 4399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf88d005c54ed384ba086c83187035" candidate_uuid: "243c29743464455aae461de5c58bf5ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc711a6bf2254e198f0fee7228c84c13" is_pre_election: true
I20260807 06:16:25.705365 4008 raft_consensus.cc:2454] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243c29743464455aae461de5c58bf5ac in term 0.
I20260807 06:16:25.705488 4399 raft_consensus.cc:2454] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243c29743464455aae461de5c58bf5ac in term 0.
I20260807 06:16:25.706148 4105 leader_election.cc:302] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 243c29743464455aae461de5c58bf5ac, e3bfc05104854c7e81666feb2612c031, fc711a6bf2254e198f0fee7228c84c13; no voters:
I20260807 06:16:25.706565 4493 raft_consensus.cc:2781] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:16:25.706681 4493 raft_consensus.cc:482] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:16:25.706764 4493 raft_consensus.cc:3037] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:25.708201 4493 raft_consensus.cc:504] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.708632 4493 leader_election.cc:288] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [CANDIDATE]: Term 1 election: Requested vote from peers e3bfc05104854c7e81666feb2612c031 (127.9.133.129:40791), fc711a6bf2254e198f0fee7228c84c13 (127.9.133.131:43391)
I20260807 06:16:25.709141 4008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf88d005c54ed384ba086c83187035" candidate_uuid: "243c29743464455aae461de5c58bf5ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bfc05104854c7e81666feb2612c031"
I20260807 06:16:25.709139 4399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf88d005c54ed384ba086c83187035" candidate_uuid: "243c29743464455aae461de5c58bf5ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc711a6bf2254e198f0fee7228c84c13"
I20260807 06:16:25.709326 4008 raft_consensus.cc:3037] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:25.709326 4399 raft_consensus.cc:3037] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:25.711181 4399 raft_consensus.cc:2454] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243c29743464455aae461de5c58bf5ac in term 1.
I20260807 06:16:25.711180 4008 raft_consensus.cc:2454] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243c29743464455aae461de5c58bf5ac in term 1.
I20260807 06:16:25.711781 4107 leader_election.cc:302] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243c29743464455aae461de5c58bf5ac, e3bfc05104854c7e81666feb2612c031; no voters:
I20260807 06:16:25.712030 4493 raft_consensus.cc:2781] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:25.712348 4493 raft_consensus.cc:686] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 1 LEADER]: Becoming Leader. State: Replica: 243c29743464455aae461de5c58bf5ac, State: Running, Role: LEADER
I20260807 06:16:25.712812 4493 consensus_queue.cc:237] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } }
I20260807 06:16:25.716985 3790 catalog_manager.cc:5495] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 243c29743464455aae461de5c58bf5ac (127.9.133.130). New cstate: current_term: 1 leader_uuid: "243c29743464455aae461de5c58bf5ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c29743464455aae461de5c58bf5ac" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 } health_report { overall_health: UNKNOWN } } }
I20260807 06:16:25.742547 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.738514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.9.133.130:34693)
0807 06:16:25.738655 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:25.738659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:25.738675 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:16:25.738857 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:25.738860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:25.738929 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:25.739088 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:16:25.739344 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.739940 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.741008 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:25.741235 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:25.741318 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:25.741611 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:16:25.742138 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:16:25.742261 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:25.742341 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 06:16:25.742386 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 06:16:25.749253 4075 tablet.cc:2369] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:16:25.750453 4008 raft_consensus.cc:1261] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 1 FOLLOWER]: Refusing update from remote peer 243c29743464455aae461de5c58bf5ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:25.750456 4399 raft_consensus.cc:1261] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 1 FOLLOWER]: Refusing update from remote peer 243c29743464455aae461de5c58bf5ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:25.751346 4493 consensus_queue.cc:1035] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc711a6bf2254e198f0fee7228c84c13" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:25.751585 4499 consensus_queue.cc:1035] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3bfc05104854c7e81666feb2612c031" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 40791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:25.759603 4503 mvcc.cc:205] Tried to move back new op lower bound from 7315797548023181312 to 7315797547883163648. Current Snapshot: MvccSnapshot[applied={T|T < 7315797548023181312}]
I20260807 06:16:25.762521 4504 mvcc.cc:205] Tried to move back new op lower bound from 7315797548023181312 to 7315797547883163648. Current Snapshot: MvccSnapshot[applied={T|T < 7315797548023181312}]
I20260807 06:16:25.773119 3792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:25.773344 3792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:25.776407 3792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e247c34e030e4c8ea2d1e667dc35a4ae: Sending DeleteTablet for 3 replicas of tablet bbbf88d005c54ed384ba086c83187035
I20260807 06:16:25.777420 4172 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbf88d005c54ed384ba086c83187035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:25 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:35560
I20260807 06:16:25.777420 4368 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbf88d005c54ed384ba086c83187035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:25 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:56764
I20260807 06:16:25.778100 4516 tablet_replica.cc:330] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: stopping tablet replica
I20260807 06:16:25.778112 3979 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbf88d005c54ed384ba086c83187035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:25 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:33800
I20260807 06:16:25.778565 4516 raft_consensus.cc:2229] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:25.778740 4522 tablet_replica.cc:330] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031: stopping tablet replica
I20260807 06:16:25.779057 4516 raft_consensus.cc:2258] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:25.779224 4522 raft_consensus.cc:2229] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:25.779523 4517 tablet_replica.cc:330] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: stopping tablet replica
I20260807 06:16:25.779680 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 3881
I20260807 06:16:25.779810 4517 raft_consensus.cc:2229] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:16:25.779884 4522 raft_consensus.cc:2258] T bbbf88d005c54ed384ba086c83187035 P e3bfc05104854c7e81666feb2612c031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:25.780316 4517 raft_consensus.cc:2258] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:25.781153 4516 ts_tablet_manager.cc:1905] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:25.786325 4516 ts_tablet_manager.cc:1918] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:25.786471 4516 log.cc:1196] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/bbbf88d005c54ed384ba086c83187035
I20260807 06:16:25.786868 4516 ts_tablet_manager.cc:1939] T bbbf88d005c54ed384ba086c83187035 P fc711a6bf2254e198f0fee7228c84c13: Deleting consensus metadata
I20260807 06:16:25.789049 3769 catalog_manager.cc:4841] TS fc711a6bf2254e198f0fee7228c84c13 (127.9.133.131:43391): tablet bbbf88d005c54ed384ba086c83187035 (table test-table [id=8a0783cb8a644a5282932ad646e81157]) successfully deleted
I20260807 06:16:25.789976 4517 ts_tablet_manager.cc:1905] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:25.793190 4517 ts_tablet_manager.cc:1918] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:25.793309 4517 log.cc:1196] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/bbbf88d005c54ed384ba086c83187035
I20260807 06:16:25.793663 4517 ts_tablet_manager.cc:1939] T bbbf88d005c54ed384ba086c83187035 P 243c29743464455aae461de5c58bf5ac: Deleting consensus metadata
I20260807 06:16:25.795255 3769 catalog_manager.cc:4841] TS 243c29743464455aae461de5c58bf5ac (127.9.133.130:34693): tablet bbbf88d005c54ed384ba086c83187035 (table test-table [id=8a0783cb8a644a5282932ad646e81157]) successfully deleted
W20260807 06:16:25.797708 3772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:40791 (error 108)
I20260807 06:16:25.798833 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 4084
I20260807 06:16:25.799120 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:25.798694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:40791 (local address 127.0.0.1:33812)
0807 06:16:25.798869 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:25.798966 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:40791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 06:16:25.806262 3772 catalog_manager.cc:4584] TS e3bfc05104854c7e81666feb2612c031 (127.9.133.129:40791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbbf88d005c54ed384ba086c83187035: Network error: Client connection negotiation failed: client connection to 127.9.133.129:40791: connect: Connection refused (error 111)
I20260807 06:16:25.813891 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 4276
I20260807 06:16:25.828030 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 3744
2026-08-07T06:16:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2887 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T06:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:25Z Disabled control of system clock
I20260807 06:16:25.869074 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:37829
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:16:26.001176 4530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:26.001519 4530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:26.001557 4530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:26.005692 4530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:26.005759 4530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:26.005774 4530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:26.005791 4530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:26.005806 4530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:26.010696 4530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:37829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:26.012233 4530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:26.014062 4530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:26.023975 4542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:26.023977 4535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:26.024204 4530 server_base.cc:1034] running on GCE node
W20260807 06:16:26.023975 4537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:26.025187 4530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:26.026732 4530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:26.027968 4530 hybrid_clock.cc:648] HybridClock initialized: now 1786083386027930 us; error 49 us; skew 500 ppm
I20260807 06:16:26.030378 4530 webserver.cc:460] Webserver started at http://127.9.133.190:33277/ using document root <none> and password file <none>
I20260807 06:16:26.031229 4530 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:26.031358 4530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:26.031649 4530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:26.034397 4530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "a808fa7ae8fb496aa000bf5beb80e4d6"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "78468c1e6f0792de21b689ed904cde61"
server_key_iv: "2b8fd65674168d0f0894de45536885e1"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.035202 4530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "a808fa7ae8fb496aa000bf5beb80e4d6"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "78468c1e6f0792de21b689ed904cde61"
server_key_iv: "2b8fd65674168d0f0894de45536885e1"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.040030 4530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:26.043172 4550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.044677 4530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:16:26.044847 4530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "a808fa7ae8fb496aa000bf5beb80e4d6"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "78468c1e6f0792de21b689ed904cde61"
server_key_iv: "2b8fd65674168d0f0894de45536885e1"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.044986 4530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:26.065750 4530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:26.066835 4530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:26.067150 4530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:26.076076 4530 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:37829
I20260807 06:16:26.076093 4618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:37829 every 8 connection(s)
I20260807 06:16:26.077417 4530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:26.081459 4619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:26.086748 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 4530
I20260807 06:16:26.086993 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:26.087426 9750 external_mini_cluster.cc:1428] Setting key 526ca634452db8f40b9ca3c7ba66f44b
I20260807 06:16:26.088919 4619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6: Bootstrap starting.
I20260807 06:16:26.091830 4619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:26.092924 4619 log.cc:824] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:26.095433 4619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6: No bootstrap required, opened a new log
I20260807 06:16:26.099116 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.089420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.9.133.190:37829)
0807 06:16:26.090223 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.090244 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.090289 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:16:26.091127 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.091151 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.091652 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.092033 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:26.092482 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.093744 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.094794 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.095462 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.095666 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.096833 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:26.096839 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:26.096854 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:26.097340 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:26.097412 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:26.097452 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:26.097588 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.097757 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.098092 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":166,"threads_started":1}
I20260807 06:16:26.100117 4619 raft_consensus.cc:348] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37829 } }
I20260807 06:16:26.100456 4619 raft_consensus.cc:374] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:26.100539 4619 raft_consensus.cc:729] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a808fa7ae8fb496aa000bf5beb80e4d6, State: Initialized, Role: FOLLOWER
I20260807 06:16:26.101054 4619 consensus_queue.cc:260] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37829 } }
I20260807 06:16:26.101194 4619 raft_consensus.cc:388] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:26.101289 4619 raft_consensus.cc:482] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:26.101426 4619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:26.102797 4619 raft_consensus.cc:504] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37829 } }
I20260807 06:16:26.103245 4619 leader_election.cc:302] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a808fa7ae8fb496aa000bf5beb80e4d6; no voters:
I20260807 06:16:26.103647 4619 leader_election.cc:288] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:26.103897 4627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:26.104216 4627 raft_consensus.cc:686] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [term 1 LEADER]: Becoming Leader. State: Replica: a808fa7ae8fb496aa000bf5beb80e4d6, State: Running, Role: LEADER
I20260807 06:16:26.104621 4627 consensus_queue.cc:237] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37829 } }
I20260807 06:16:26.105320 4619 sys_catalog.cc:564] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:26.106765 4631 sys_catalog.cc:455] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a808fa7ae8fb496aa000bf5beb80e4d6. Latest consensus state: current_term: 1 leader_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37829 } } }
I20260807 06:16:26.106819 4630 sys_catalog.cc:455] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a808fa7ae8fb496aa000bf5beb80e4d6" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37829 } } }
I20260807 06:16:26.106913 4631 sys_catalog.cc:458] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:26.106936 4630 sys_catalog.cc:458] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:26.107594 4644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:26.111799 4644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:26.119179 4644 catalog_manager.cc:1347] Generated new cluster ID: bc30a9d07c474b1caceb201cfb576353
I20260807 06:16:26.119302 4644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:26.132949 4644 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:26.134294 4644 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:26.145386 4644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6: Generated new TSK 0
I20260807 06:16:26.146456 4644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:26.160193 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:37829
--builtin_ntp_servers=127.9.133.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:16:26.294066 4653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:26.294435 4653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:26.294514 4653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:26.298650 4653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:26.298756 4653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:26.298857 4653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:26.304001 4653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:37829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:26.305434 4653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:26.307404 4653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:26.319023 4659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:26.319015 4660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:26.319185 4653 server_base.cc:1034] running on GCE node
W20260807 06:16:26.319411 4662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:26.319900 4653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:26.320652 4653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:26.321836 4653 hybrid_clock.cc:648] HybridClock initialized: now 1786083386321817 us; error 32 us; skew 500 ppm
I20260807 06:16:26.324190 4653 webserver.cc:460] Webserver started at http://127.9.133.129:36389/ using document root <none> and password file <none>
I20260807 06:16:26.324921 4653 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:26.325023 4653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:26.325304 4653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:26.327916 4653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "ae8e08b380a14346b4e362d6f6878821"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "e05af24c732878a6d281c98c1c244d9c"
server_key_iv: "80e30e432a0435f67ab9f08367226242"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.328559 4653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "ae8e08b380a14346b4e362d6f6878821"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "e05af24c732878a6d281c98c1c244d9c"
server_key_iv: "80e30e432a0435f67ab9f08367226242"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.333144 4653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:26.336390 4670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.338068 4653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:16:26.338264 4653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "ae8e08b380a14346b4e362d6f6878821"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "e05af24c732878a6d281c98c1c244d9c"
server_key_iv: "80e30e432a0435f67ab9f08367226242"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.338431 4653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:26.357312 4653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:26.358273 4653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:26.358536 4653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:26.359541 4653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:26.360889 4653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:26.360947 4653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.361029 4653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:26.361065 4653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.372756 4653 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:38505
I20260807 06:16:26.372789 4823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:38505 every 8 connection(s)
I20260807 06:16:26.373932 4653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:26.380195 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 4653
I20260807 06:16:26.380349 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:16:26.380712 9750 external_mini_cluster.cc:1428] Setting key ca70d8665902528cf8abe3a6360e67b6
I20260807 06:16:26.383241 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:37829
--builtin_ntp_servers=127.9.133.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:16:26.386240 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.376273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:50347 (local address 127.9.133.190:37829)
0807 06:16:26.376456 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.376461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.377790 (+ 1329us) server_negotiation.cc:407] Connection header received
0807 06:16:26.379611 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.379615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.379689 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.379807 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:26.381031 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.381830 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.383671 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.383915 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.384014 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.385562 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:26.385569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:26.385575 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:26.385595 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:26.385629 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:26.385643 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:26.385744 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.385930 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.386015 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:16:26.387231 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.376755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:37829 (local address 127.9.133.129:50347)
0807 06:16:26.377593 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:26.377643 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:26.379357 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:26.380026 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:26.380035 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:26.380137 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:26.380872 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.380890 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.381970 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.381973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.383514 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.383530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.384099 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.384106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.384365 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.385168 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:26.385173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:26.385240 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:26.385339 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:26.385759 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:26.385774 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:26.385780 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:16:26.385895 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 06:16:26.386162 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":146,"threads_started":1}
I20260807 06:16:26.388633 4824 heartbeater.cc:347] Connected to a master server at 127.9.133.190:37829
I20260807 06:16:26.388913 4824 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:26.389567 4824 heartbeater.cc:511] Master 127.9.133.190:37829 requested a full tablet report, sending...
I20260807 06:16:26.391438 4570 ts_manager.cc:194] Registered new tserver with Master: ae8e08b380a14346b4e362d6f6878821 (127.9.133.129:38505)
I20260807 06:16:26.392807 4570 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.129:50347
W20260807 06:16:26.518210 4827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:26.518498 4827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:26.518539 4827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:26.522677 4827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:26.522733 4827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:26.522832 4827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:26.527717 4827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:37829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:26.529034 4827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:26.530764 4827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:26.545920 4834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:26.545913 4837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:26.545939 4835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:26.546522 4827 server_base.cc:1034] running on GCE node
I20260807 06:16:26.546981 4827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:26.547827 4827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:26.549027 4827 hybrid_clock.cc:648] HybridClock initialized: now 1786083386548992 us; error 55 us; skew 500 ppm
I20260807 06:16:26.551415 4827 webserver.cc:460] Webserver started at http://127.9.133.130:39115/ using document root <none> and password file <none>
I20260807 06:16:26.552134 4827 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:26.552250 4827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:26.552505 4827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:26.555168 4827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "0546e72d49c12d7eb88a9d5ae04692a4"
server_key_iv: "31f06e316974f16630a9cbe98b9f879a"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.555958 4827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "0546e72d49c12d7eb88a9d5ae04692a4"
server_key_iv: "31f06e316974f16630a9cbe98b9f879a"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.561321 4827 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 06:16:26.564363 4847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.565955 4827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:26.566120 4827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "0546e72d49c12d7eb88a9d5ae04692a4"
server_key_iv: "31f06e316974f16630a9cbe98b9f879a"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.566247 4827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:26.585803 4827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:26.586771 4827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:26.587081 4827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:26.587869 4827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:26.589120 4827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:26.589218 4827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.589320 4827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:26.589371 4827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.602126 4827 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:40871
I20260807 06:16:26.602206 5000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:40871 every 8 connection(s)
I20260807 06:16:26.603389 4827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:26.613569 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 4827
I20260807 06:16:26.613730 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:16:26.614076 9750 external_mini_cluster.cc:1428] Setting key 2f6ccd0763eb075492a0b770ca6cb88e
I20260807 06:16:26.616904 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:37829
--builtin_ntp_servers=127.9.133.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:16:26.618043 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.606046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:34049 (local address 127.9.133.190:37829)
0807 06:16:26.606281 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.606289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.608076 (+ 1787us) server_negotiation.cc:407] Connection header received
0807 06:16:26.610655 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.610659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.610728 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.610816 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:26.612552 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.613163 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.615483 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.615826 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.615950 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.617456 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:26.617460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:26.617463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:26.617485 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:26.617512 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:26.617521 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:26.617601 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.617787 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.617866 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 06:16:26.619318 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.606730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:37829 (local address 127.9.133.130:34049)
0807 06:16:26.607798 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:26.607871 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:26.610279 (+ 2408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:26.611151 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:26.611167 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:26.611333 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:26.612320 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.612346 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.613326 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.613332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.615335 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.615351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.616008 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.616014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.616244 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.617072 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:26.617075 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:26.617159 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:26.617277 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:26.617603 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:26.617617 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:26.617624 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:16:26.617775 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 06:16:26.618171 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":223,"threads_started":1}
I20260807 06:16:26.620872 5001 heartbeater.cc:347] Connected to a master server at 127.9.133.190:37829
I20260807 06:16:26.621198 5001 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:26.621886 5001 heartbeater.cc:511] Master 127.9.133.190:37829 requested a full tablet report, sending...
I20260807 06:16:26.623349 4570 ts_manager.cc:194] Registered new tserver with Master: 4eecda2ff93a460fa2bb9d0f9534fcc5 (127.9.133.130:40871)
I20260807 06:16:26.624392 4570 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.130:34049
W20260807 06:16:26.753173 5005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:26.753520 5005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:26.753593 5005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:26.757867 5005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:26.757953 5005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:26.758044 5005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:26.763135 5005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:37829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:26.764506 5005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:26.766247 5005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:26.778052 5011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:26.778056 5015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:26.778052 5010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:26.778532 5005 server_base.cc:1034] running on GCE node
I20260807 06:16:26.779105 5005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:26.779901 5005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:26.781123 5005 hybrid_clock.cc:648] HybridClock initialized: now 1786083386781095 us; error 49 us; skew 500 ppm
I20260807 06:16:26.783478 5005 webserver.cc:460] Webserver started at http://127.9.133.131:36791/ using document root <none> and password file <none>
I20260807 06:16:26.784241 5005 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:26.784344 5005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:26.784623 5005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:26.787230 5005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "741f3d4cc9d44100a3c4a929c2d6a152"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "2ad2591c90405d0b886d2d5c1d50bbf1"
server_key_iv: "bd7a8523f4c171d8c45211a567605f67"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.787938 5005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "741f3d4cc9d44100a3c4a929c2d6a152"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "2ad2591c90405d0b886d2d5c1d50bbf1"
server_key_iv: "bd7a8523f4c171d8c45211a567605f67"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.792452 5005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:26.795506 5021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.797076 5005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:16:26.797230 5005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "741f3d4cc9d44100a3c4a929c2d6a152"
format_stamp: "Formatted at 2026-08-07 06:16:26 on dist-test-slave-h3ft"
server_key: "2ad2591c90405d0b886d2d5c1d50bbf1"
server_key_iv: "bd7a8523f4c171d8c45211a567605f67"
server_key_version: "encryptionkey@0"
I20260807 06:16:26.797367 5005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:26.817816 5005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:26.818770 5005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:26.819039 5005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:26.819795 5005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:26.821141 5005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:26.821295 5005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.821439 5005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:26.821499 5005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:26.832708 5005 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:35961
I20260807 06:16:26.832733 5190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:35961 every 8 connection(s)
I20260807 06:16:26.833877 5005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:26.838959 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 5005
I20260807 06:16:26.839159 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:16:26.839558 9750 external_mini_cluster.cc:1428] Setting key 00f87336ba6a7721a2470776377a91db
I20260807 06:16:26.846127 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.836264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:42811 (local address 127.9.133.190:37829)
0807 06:16:26.836432 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.836437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.837723 (+ 1286us) server_negotiation.cc:407] Connection header received
0807 06:16:26.839439 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.839442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.839522 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.839645 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:26.840938 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.841530 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.843439 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.843889 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.844039 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.845575 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:26.845578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:26.845581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:26.845596 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:26.845620 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:26.845629 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:26.845674 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.845890 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.845959 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:16:26.847821 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.836714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:37829 (local address 127.9.133.131:42811)
0807 06:16:26.837536 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:26.837583 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:26.839168 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:26.839850 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:26.839858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:26.839955 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:26.840780 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.840803 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.841662 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.841666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.843276 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.843290 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.844104 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.844109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.844387 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.845217 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:26.845219 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:26.845281 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:26.845377 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:26.845734 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:26.845743 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:26.845749 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:16:26.846302 (+ 553us) client_negotiation.cc:240] Negotiation successful
0807 06:16:26.846645 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":195,"threads_started":1}
I20260807 06:16:26.854552 5191 heartbeater.cc:347] Connected to a master server at 127.9.133.190:37829
I20260807 06:16:26.854880 5191 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:26.855795 5191 heartbeater.cc:511] Master 127.9.133.190:37829 requested a full tablet report, sending...
I20260807 06:16:26.857357 4570 ts_manager.cc:194] Registered new tserver with Master: 741f3d4cc9d44100a3c4a929c2d6a152 (127.9.133.131:35961)
I20260807 06:16:26.858266 4570 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.131:42811
I20260807 06:16:26.867766 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:16:26.874814 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.870226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.9.133.190:37829)
0807 06:16:26.870391 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.870397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.870567 (+ 170us) server_negotiation.cc:407] Connection header received
0807 06:16:26.870738 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.870741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.870801 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.870892 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:26.871272 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.871900 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.872840 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.873091 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.873206 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.874248 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:26.874252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:26.874255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:26.874270 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:26.874302 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:26.874311 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:26.874460 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.874586 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.874632 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 06:16:26.878294 4570 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:16:26.881927 4570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:16:26.898797 5212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.893314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:40871 (local address 127.0.0.1:44654)
0807 06:16:26.893709 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:26.893724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:26.893835 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:26.894647 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:26.894649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:26.894668 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:26.894756 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.894762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.896159 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.896163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.897092 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.897102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.898112 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.898116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.898244 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.898320 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 06:16:26.898432 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 06:16:26.898607 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":53,"thread_start_us":123,"threads_started":1}
I20260807 06:16:26.899329 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.892115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:35961 (local address 127.0.0.1:51566)
0807 06:16:26.892933 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:26.893002 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:26.893153 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:26.894049 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:26.894055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:26.894090 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:26.894457 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.894465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.896100 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.896103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.897544 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.897555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.898866 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.898870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.898967 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.898992 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:16:26.899087 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 06:16:26.899159 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":284,"spinlock_wait_cycles":10496,"thread_start_us":102,"threads_started":1}
I20260807 06:16:26.899622 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.893658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.9.133.130:40871)
0807 06:16:26.894109 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.894147 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.894175 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:16:26.894265 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.894270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.894480 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.894667 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:26.894845 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.896021 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.897232 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.897982 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.898092 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.898321 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.899269 (+ 948us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.899395 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":150,"threads_started":1}
I20260807 06:16:26.899762 5206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.892251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38505 (local address 127.0.0.1:60626)
0807 06:16:26.892933 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:26.892995 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:26.893153 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:26.894049 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:26.894055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:26.894091 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:26.894339 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.894347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.895974 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.895979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.897544 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:26.897555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.899365 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.899369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:26.899455 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.899482 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 06:16:26.899549 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 06:16:26.899621 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":160,"spinlock_wait_cycles":11776,"thread_start_us":103,"threads_started":1}
I20260807 06:16:26.899892 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.893017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.9.133.129:38505)
0807 06:16:26.893478 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.893501 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.893526 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:16:26.893616 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.893622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.893856 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.894186 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:26.894471 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.895852 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.898399 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.899243 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.899363 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.899571 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.899625 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.899732 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":149,"threads_started":1}
I20260807 06:16:26.900539 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:26.892326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.9.133.131:35961)
0807 06:16:26.893478 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:26.893492 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:26.893523 (+ 31us) server_negotiation.cc:407] Connection header received
0807 06:16:26.893616 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:26.893622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:26.893856 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:26.894217 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:26.894614 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.895918 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.897688 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:26.898505 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:26.898604 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:26.898799 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:26.899627 (+ 828us) server_negotiation.cc:299] Negotiation successful
0807 06:16:26.900361 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":180,"threads_started":1}
I20260807 06:16:26.903286 4905 tablet_service.cc:1467] Processing CreateTablet for tablet b1a353c031b141bc9b61d17413557d5f (DEFAULT_TABLE table=test-table [id=50d62ec11ef34185aee30a5159b5a3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:26.903290 5092 tablet_service.cc:1467] Processing CreateTablet for tablet b1a353c031b141bc9b61d17413557d5f (DEFAULT_TABLE table=test-table [id=50d62ec11ef34185aee30a5159b5a3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:26.903291 4731 tablet_service.cc:1467] Processing CreateTablet for tablet b1a353c031b141bc9b61d17413557d5f (DEFAULT_TABLE table=test-table [id=50d62ec11ef34185aee30a5159b5a3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:26.904507 4905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a353c031b141bc9b61d17413557d5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:26.904515 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a353c031b141bc9b61d17413557d5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:26.904511 4731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a353c031b141bc9b61d17413557d5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:26.912199 5215 tablet_bootstrap.cc:492] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Bootstrap starting.
I20260807 06:16:26.913364 5216 tablet_bootstrap.cc:492] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: Bootstrap starting.
I20260807 06:16:26.913892 5219 tablet_bootstrap.cc:492] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Bootstrap starting.
I20260807 06:16:26.915905 5219 tablet_bootstrap.cc:654] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:26.916047 5216 tablet_bootstrap.cc:654] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:26.916998 5216 log.cc:824] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:26.916996 5219 log.cc:824] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:26.917092 5215 tablet_bootstrap.cc:654] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:26.917887 5215 log.cc:824] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:26.919471 5216 tablet_bootstrap.cc:492] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: No bootstrap required, opened a new log
I20260807 06:16:26.919471 5219 tablet_bootstrap.cc:492] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: No bootstrap required, opened a new log
I20260807 06:16:26.919697 5219 ts_tablet_manager.cc:1397] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 06:16:26.919697 5216 ts_tablet_manager.cc:1397] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 06:16:26.919715 5215 tablet_bootstrap.cc:492] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: No bootstrap required, opened a new log
I20260807 06:16:26.919917 5215 ts_tablet_manager.cc:1397] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 06:16:26.923669 5215 raft_consensus.cc:348] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:26.923949 5215 raft_consensus.cc:374] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:26.924008 5215 raft_consensus.cc:729] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eecda2ff93a460fa2bb9d0f9534fcc5, State: Initialized, Role: FOLLOWER
I20260807 06:16:26.924525 5215 consensus_queue.cc:260] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:26.924635 5216 raft_consensus.cc:348] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:26.924625 5219 raft_consensus.cc:348] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:26.924887 5216 raft_consensus.cc:374] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:26.924887 5219 raft_consensus.cc:374] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:26.924952 5219 raft_consensus.cc:729] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741f3d4cc9d44100a3c4a929c2d6a152, State: Initialized, Role: FOLLOWER
I20260807 06:16:26.924962 5216 raft_consensus.cc:729] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae8e08b380a14346b4e362d6f6878821, State: Initialized, Role: FOLLOWER
I20260807 06:16:26.925468 5001 heartbeater.cc:503] Master 127.9.133.190:37829 was elected leader, sending a full tablet report...
I20260807 06:16:26.925400 5219 consensus_queue.cc:260] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:26.925894 5215 ts_tablet_manager.cc:1428] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260807 06:16:26.926131 5191 heartbeater.cc:503] Master 127.9.133.190:37829 was elected leader, sending a full tablet report...
I20260807 06:16:26.926379 5219 ts_tablet_manager.cc:1428] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 06:16:26.927175 5216 consensus_queue.cc:260] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:26.927876 4824 heartbeater.cc:503] Master 127.9.133.190:37829 was elected leader, sending a full tablet report...
I20260807 06:16:26.928252 5216 ts_tablet_manager.cc:1428] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 06:16:27.060177 5226 raft_consensus.cc:482] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:27.060549 5226 raft_consensus.cc:504] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:27.062110 5226 leader_election.cc:288] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae8e08b380a14346b4e362d6f6878821 (127.9.133.129:38505), 4eecda2ff93a460fa2bb9d0f9534fcc5 (127.9.133.130:40871)
I20260807 06:16:27.068542 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:27.062970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:56895 (local address 127.9.133.129:38505)
0807 06:16:27.063160 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:27.063164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:27.063179 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:16:27.063267 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:27.063270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:27.063336 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:27.063426 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:27.063988 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.064819 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.067090 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.067738 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.067856 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:27.067920 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:27.068278 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 06:16:27.068367 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 06:16:27.068540 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:27.062690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:37339 (local address 127.9.133.130:40871)
0807 06:16:27.062881 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:27.062888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:27.063279 (+ 391us) server_negotiation.cc:407] Connection header received
0807 06:16:27.063382 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:27.063386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:27.063456 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:27.063538 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:27.063943 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.064819 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.067068 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.067714 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.067856 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:27.067937 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:27.068278 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 06:16:27.068368 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 06:16:27.068588 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:27.062500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38505 (local address 127.9.133.131:56895)
0807 06:16:27.062666 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:27.062686 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:27.062814 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:27.063494 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:27.063497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:27.063556 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:27.063782 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:27.063791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.064987 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.064992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:27.066933 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:27.066944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.067939 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.067943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:27.068115 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:27.068145 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 06:16:27.068243 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 06:16:27.068381 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":367,"spinlock_wait_cycles":22912}
I20260807 06:16:27.068598 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:27.062504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:40871 (local address 127.9.133.131:37339)
0807 06:16:27.063155 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:27.063167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:27.063280 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:27.063584 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:27.063586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:27.063631 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:27.063790 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:27.063796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.064987 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.064992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:27.066868 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:27.066882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.067943 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.067947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:27.068100 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:27.068149 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 06:16:27.068243 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:16:27.068388 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":361,"spinlock_wait_cycles":384,"thread_start_us":252,"threads_started":1}
I20260807 06:16:27.069563 4937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a353c031b141bc9b61d17413557d5f" candidate_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" is_pre_election: true
I20260807 06:16:27.069581 4759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a353c031b141bc9b61d17413557d5f" candidate_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae8e08b380a14346b4e362d6f6878821" is_pre_election: true
I20260807 06:16:27.070007 4759 raft_consensus.cc:2454] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741f3d4cc9d44100a3c4a929c2d6a152 in term 0.
I20260807 06:16:27.070008 4937 raft_consensus.cc:2454] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741f3d4cc9d44100a3c4a929c2d6a152 in term 0.
I20260807 06:16:27.070636 5022 leader_election.cc:302] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eecda2ff93a460fa2bb9d0f9534fcc5, 741f3d4cc9d44100a3c4a929c2d6a152; no voters:
I20260807 06:16:27.071185 5226 raft_consensus.cc:2781] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:16:27.071291 5226 raft_consensus.cc:482] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:16:27.071348 5226 raft_consensus.cc:3037] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:27.072649 5226 raft_consensus.cc:504] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:27.073076 5226 leader_election.cc:288] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [CANDIDATE]: Term 1 election: Requested vote from peers ae8e08b380a14346b4e362d6f6878821 (127.9.133.129:38505), 4eecda2ff93a460fa2bb9d0f9534fcc5 (127.9.133.130:40871)
I20260807 06:16:27.073606 4937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a353c031b141bc9b61d17413557d5f" candidate_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5"
I20260807 06:16:27.073712 4759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a353c031b141bc9b61d17413557d5f" candidate_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae8e08b380a14346b4e362d6f6878821"
I20260807 06:16:27.073740 4937 raft_consensus.cc:3037] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:27.073886 4759 raft_consensus.cc:3037] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:27.075520 4937 raft_consensus.cc:2454] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741f3d4cc9d44100a3c4a929c2d6a152 in term 1.
I20260807 06:16:27.076058 4759 raft_consensus.cc:2454] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741f3d4cc9d44100a3c4a929c2d6a152 in term 1.
I20260807 06:16:27.076059 5022 leader_election.cc:302] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eecda2ff93a460fa2bb9d0f9534fcc5, 741f3d4cc9d44100a3c4a929c2d6a152; no voters:
I20260807 06:16:27.076256 5226 raft_consensus.cc:2781] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:27.076546 5226 raft_consensus.cc:686] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 1 LEADER]: Becoming Leader. State: Replica: 741f3d4cc9d44100a3c4a929c2d6a152, State: Running, Role: LEADER
I20260807 06:16:27.076977 5226 consensus_queue.cc:237] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } }
I20260807 06:16:27.080915 4570 catalog_manager.cc:5495] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 741f3d4cc9d44100a3c4a929c2d6a152 (127.9.133.131). New cstate: current_term: 1 leader_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741f3d4cc9d44100a3c4a929c2d6a152" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 } health_report { overall_health: UNKNOWN } } }
W20260807 06:16:27.085042 5193 tablet.cc:2369] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:16:27.105203 5002 tablet.cc:2369] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 06:16:27.125732 4825 tablet.cc:2369] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:16:27.140246 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:27.136093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.9.133.131:35961)
0807 06:16:27.136250 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:27.136255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:27.136271 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:16:27.136386 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:27.136389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:27.136463 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:27.136562 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:16:27.136888 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.137466 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.138495 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:27.138720 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:27.138803 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:27.139250 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:16:27.139806 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:16:27.139978 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:27.140061 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 06:16:27.140098 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 06:16:27.148653 4937 raft_consensus.cc:1261] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 1 FOLLOWER]: Refusing update from remote peer 741f3d4cc9d44100a3c4a929c2d6a152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:27.148653 4759 raft_consensus.cc:1261] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 1 FOLLOWER]: Refusing update from remote peer 741f3d4cc9d44100a3c4a929c2d6a152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:27.149757 5226 consensus_queue.cc:1035] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eecda2ff93a460fa2bb9d0f9534fcc5" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:27.150035 5231 consensus_queue.cc:1035] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae8e08b380a14346b4e362d6f6878821" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 38505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:27.161974 5237 mvcc.cc:205] Tried to move back new op lower bound from 7315797553750380544 to 7315797553470382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315797553750380544}]
I20260807 06:16:27.163100 5236 mvcc.cc:205] Tried to move back new op lower bound from 7315797553750380544 to 7315797553470382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315797553750380544 or (T in {7315797553750380544})}]
I20260807 06:16:27.170670 4570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:27.170964 4570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:27.174432 4570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a808fa7ae8fb496aa000bf5beb80e4d6: Sending DeleteTablet for 3 replicas of tablet b1a353c031b141bc9b61d17413557d5f
I20260807 06:16:27.175467 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet b1a353c031b141bc9b61d17413557d5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:27 UTC) from {username='slave'} at 127.0.0.1:51566
I20260807 06:16:27.175545 4905 tablet_service.cc:1514] Processing DeleteTablet for tablet b1a353c031b141bc9b61d17413557d5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:27 UTC) from {username='slave'} at 127.0.0.1:44654
I20260807 06:16:27.175560 4731 tablet_service.cc:1514] Processing DeleteTablet for tablet b1a353c031b141bc9b61d17413557d5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:27 UTC) from {username='slave'} at 127.0.0.1:60626
I20260807 06:16:27.176211 5251 tablet_replica.cc:330] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: stopping tablet replica
I20260807 06:16:27.176476 5252 tablet_replica.cc:330] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821: stopping tablet replica
I20260807 06:16:27.176698 5251 raft_consensus.cc:2229] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:16:27.177003 5252 raft_consensus.cc:2229] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:27.177264 5251 raft_consensus.cc:2258] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:27.177361 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 4653
I20260807 06:16:27.177568 5252 raft_consensus.cc:2258] T b1a353c031b141bc9b61d17413557d5f P ae8e08b380a14346b4e362d6f6878821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:27.177727 5250 tablet_replica.cc:330] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: stopping tablet replica
I20260807 06:16:27.178097 5250 raft_consensus.cc:2229] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:27.178454 5250 raft_consensus.cc:2258] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:27.182828 5250 ts_tablet_manager.cc:1905] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:27.186311 5250 ts_tablet_manager.cc:1918] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:27.186447 5250 log.cc:1196] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/b1a353c031b141bc9b61d17413557d5f
I20260807 06:16:27.186807 5250 ts_tablet_manager.cc:1939] T b1a353c031b141bc9b61d17413557d5f P 4eecda2ff93a460fa2bb9d0f9534fcc5: Deleting consensus metadata
I20260807 06:16:27.188529 4552 catalog_manager.cc:4841] TS 4eecda2ff93a460fa2bb9d0f9534fcc5 (127.9.133.130:40871): tablet b1a353c031b141bc9b61d17413557d5f (table test-table [id=50d62ec11ef34185aee30a5159b5a3f0]) successfully deleted
I20260807 06:16:27.188974 5251 ts_tablet_manager.cc:1905] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:27.193024 5251 ts_tablet_manager.cc:1918] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:27.193253 5251 log.cc:1196] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/b1a353c031b141bc9b61d17413557d5f
I20260807 06:16:27.193696 5251 ts_tablet_manager.cc:1939] T b1a353c031b141bc9b61d17413557d5f P 741f3d4cc9d44100a3c4a929c2d6a152: Deleting consensus metadata
I20260807 06:16:27.195411 4553 catalog_manager.cc:4841] TS 741f3d4cc9d44100a3c4a929c2d6a152 (127.9.133.131:35961): tablet b1a353c031b141bc9b61d17413557d5f (table test-table [id=50d62ec11ef34185aee30a5159b5a3f0]) successfully deleted
W20260807 06:16:27.196483 4554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:38505 (error 108)
I20260807 06:16:27.196826 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 4827
I20260807 06:16:27.202435 5206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:27.201844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:38505 (local address 127.0.0.1:60634)
0807 06:16:27.202159 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:27.202280 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:38505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260807 06:16:27.202723 4554 catalog_manager.cc:4584] TS ae8e08b380a14346b4e362d6f6878821 (127.9.133.129:38505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1a353c031b141bc9b61d17413557d5f: Network error: Client connection negotiation failed: client connection to 127.9.133.129:38505: connect: Connection refused (error 111)
I20260807 06:16:27.219790 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 5005
I20260807 06:16:27.233589 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 4530
2026-08-07T06:16:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1406 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:27 dist-test-slave-h3ft krb5kdc[5256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:27 dist-test-slave-h3ft krb5kdc[5256](info): set up 2 sockets
Aug 07 06:16:27 dist-test-slave-h3ft krb5kdc[5256](info): commencing operation
krb5kdc: starting...
W20260807 06:16:28.522455 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:28 dist-test-slave-h3ft krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:28.678885 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:33757
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:33757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:28.810889 5276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:28.811331 5276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:28.811415 5276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:28.816171 5276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:28.816318 5276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:28.816367 5276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:28.816414 5276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:28.816455 5276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:28.822535 5276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:33757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:33757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:28.824041 5276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:28.825444 5276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:28.838944 5282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:28.838944 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:28.839104 5281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:28.839814 5276 server_base.cc:1034] running on GCE node
I20260807 06:16:28.840844 5276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:28.842950 5276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:28.844261 5276 hybrid_clock.cc:648] HybridClock initialized: now 1786083388844214 us; error 49 us; skew 500 ppm
Aug 07 06:16:28 dist-test-slave-h3ft krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083388, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:28.849043 5276 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:28.851755 5276 webserver.cc:460] Webserver started at http://127.9.133.190:43457/ using document root <none> and password file <none>
I20260807 06:16:28.852799 5276 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:28.852955 5276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:28.853320 5276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:28.856799 5276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "e56f6a1b079b410984c9be872cfef9a9"
format_stamp: "Formatted at 2026-08-07 06:16:28 on dist-test-slave-h3ft"
server_key: "9ca9236a58de7fac98095e0d7a2250df"
server_key_iv: "77dad1328527ea574708a736f0e623b2"
server_key_version: "encryptionkey@0"
I20260807 06:16:28.857556 5276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "e56f6a1b079b410984c9be872cfef9a9"
format_stamp: "Formatted at 2026-08-07 06:16:28 on dist-test-slave-h3ft"
server_key: "9ca9236a58de7fac98095e0d7a2250df"
server_key_iv: "77dad1328527ea574708a736f0e623b2"
server_key_version: "encryptionkey@0"
I20260807 06:16:28.867719 5276 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.002s sys 0.008s
I20260807 06:16:28.871982 5294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:28.875870 5276 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260807 06:16:28.876114 5276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "e56f6a1b079b410984c9be872cfef9a9"
format_stamp: "Formatted at 2026-08-07 06:16:28 on dist-test-slave-h3ft"
server_key: "9ca9236a58de7fac98095e0d7a2250df"
server_key_iv: "77dad1328527ea574708a736f0e623b2"
server_key_version: "encryptionkey@0"
I20260807 06:16:28.876308 5276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:28.895685 5276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:28.899717 5276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:28.899991 5276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:28.908553 5276 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:33757
I20260807 06:16:28.908577 5364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:33757 every 8 connection(s)
I20260807 06:16:28.909708 5276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:28.913396 5365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:28.918414 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 5276
I20260807 06:16:28.918624 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:28.918964 9750 external_mini_cluster.cc:1428] Setting key b683094072f45586b223742750087af5
I20260807 06:16:28.920868 5365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: Bootstrap starting.
I20260807 06:16:28.923817 5365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:28.924769 5365 log.cc:824] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:28.927121 5365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: No bootstrap required, opened a new log
Aug 07 06:16:28 dist-test-slave-h3ft krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:28.932204 5365 raft_consensus.cc:348] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56f6a1b079b410984c9be872cfef9a9" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33757 } }
I20260807 06:16:28.932528 5365 raft_consensus.cc:374] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:28.932644 5365 raft_consensus.cc:729] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56f6a1b079b410984c9be872cfef9a9, State: Initialized, Role: FOLLOWER
I20260807 06:16:28.933276 5365 consensus_queue.cc:260] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56f6a1b079b410984c9be872cfef9a9" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33757 } }
I20260807 06:16:28.933465 5365 raft_consensus.cc:388] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:28.933566 5365 raft_consensus.cc:482] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:28.933688 5365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:28.934959 5365 raft_consensus.cc:504] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56f6a1b079b410984c9be872cfef9a9" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33757 } }
I20260807 06:16:28.935459 5365 leader_election.cc:302] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e56f6a1b079b410984c9be872cfef9a9; no voters:
I20260807 06:16:28.935858 5365 leader_election.cc:288] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:28.936008 5373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:28.936086 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:28.920856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34416 (local address 127.9.133.190:33757)
0807 06:16:28.921415 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:28.921434 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:28.921479 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:16:28.922240 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:28.922262 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:28.922671 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:28.923094 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:28.924080 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:28.925247 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:28.926175 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:28.926695 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:28.926904 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:28.930038 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:28.930044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:28.930061 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:28.930093 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:28.932218 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:28.932765 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:28.932777 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:28.932783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:28.932916 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:28.933233 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:28.933238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:28.933240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:28.933881 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:28.934142 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:28.934432 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 06:16:28.934825 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":174,"threads_started":1}
I20260807 06:16:28.936261 5373 raft_consensus.cc:686] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [term 1 LEADER]: Becoming Leader. State: Replica: e56f6a1b079b410984c9be872cfef9a9, State: Running, Role: LEADER
I20260807 06:16:28.936679 5373 consensus_queue.cc:237] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56f6a1b079b410984c9be872cfef9a9" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33757 } }
I20260807 06:16:28.937573 5365 sys_catalog.cc:564] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:28.940336 5375 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e56f6a1b079b410984c9be872cfef9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56f6a1b079b410984c9be872cfef9a9" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33757 } } }
I20260807 06:16:28.940398 5376 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e56f6a1b079b410984c9be872cfef9a9. Latest consensus state: current_term: 1 leader_uuid: "e56f6a1b079b410984c9be872cfef9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56f6a1b079b410984c9be872cfef9a9" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 33757 } } }
I20260807 06:16:28.940874 5375 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:28.940876 5376 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9 [sys.catalog]: This master's current role is: LEADER
W20260807 06:16:28.943949 5393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:28.944026 5393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:28.944115 5394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:28.944900 5394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:28.952136 5394 catalog_manager.cc:1347] Generated new cluster ID: 833f699fab88417cb66642b3f778cca6
I20260807 06:16:28.952219 5394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:28.964028 5394 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:28.965258 5394 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:28.971757 5394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: Generated new TSK 0
I20260807 06:16:28.972591 5394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.133.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
Entry for principal kudu/127.9.133.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab.
I20260807 06:16:29.019548 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:33757
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:29.161878 5400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:29.162214 5400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:29.162261 5400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:29.166522 5400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:29.166594 5400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:29.166656 5400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:29.172367 5400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.129.keytab
--principal=kudu/127.9.133.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:29.173681 5400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:29.174883 5400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:29.186494 5418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:29.186494 5410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:29.186713 5400 server_base.cc:1034] running on GCE node
W20260807 06:16:29.186558 5409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:29.187368 5400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:29.188133 5400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:29.189303 5400 hybrid_clock.cc:648] HybridClock initialized: now 1786083389189281 us; error 34 us; skew 500 ppm
Aug 07 06:16:29 dist-test-slave-h3ft krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083389, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:29.192662 5400 init.cc:377] Logged in from keytab as kudu/127.9.133.129@KRBTEST.COM (short username kudu)
I20260807 06:16:29.193619 5400 webserver.cc:460] Webserver started at http://127.9.133.129:34455/ using document root <none> and password file <none>
I20260807 06:16:29.194272 5400 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:29.194339 5400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:29.194540 5400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:29.196967 5400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "89392ca71bad4e3d8508306b6293d4cb"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "c2d6dc2b5db9ce7eaf05a9293ce90fa1"
server_key_iv: "9c8792a834cfaf9563f73c4b05d5c753"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.197611 5400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "89392ca71bad4e3d8508306b6293d4cb"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "c2d6dc2b5db9ce7eaf05a9293ce90fa1"
server_key_iv: "9c8792a834cfaf9563f73c4b05d5c753"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.202132 5400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 06:16:29.205013 5428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.206403 5400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 06:16:29.206522 5400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "89392ca71bad4e3d8508306b6293d4cb"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "c2d6dc2b5db9ce7eaf05a9293ce90fa1"
server_key_iv: "9c8792a834cfaf9563f73c4b05d5c753"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.206624 5400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:29.232609 5400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:29.236344 5400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:29.236527 5400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:29.237214 5400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:29.238260 5400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:29.238312 5400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.238358 5400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:29.238374 5400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.249828 5400 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:42201
I20260807 06:16:29.249866 5600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:42201 every 8 connection(s)
I20260807 06:16:29.251196 5400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:29.260667 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 5400
I20260807 06:16:29.260838 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:16:29.261204 9750 external_mini_cluster.cc:1428] Setting key e8fcf6017793e454852f830316c3258b
Aug 07 06:16:29 dist-test-slave-h3ft krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083389, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.129@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:29.267810 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.253689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:37929 (local address 127.9.133.190:33757)
0807 06:16:29.253855 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.253859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.254828 (+ 969us) server_negotiation.cc:407] Connection header received
0807 06:16:29.256423 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.256427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.256486 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.256568 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:29.257927 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.258517 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.259795 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.260042 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.260126 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.263118 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:29.263122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:29.263124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:29.263155 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:29.265133 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.265799 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.265805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.265807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.265915 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.266295 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.266301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.266303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.266590 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:29.266708 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.267360 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.267531 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:16:29.268697 5603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.254094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:33757 (local address 127.9.133.129:37929)
0807 06:16:29.254628 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.254678 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.256195 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.256757 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.256765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.257218 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:29.257783 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.257796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.258631 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.258634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.259679 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.259688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.260198 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.260201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.260363 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.261117 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:29.261135 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:29.262884 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:29.265273 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:29.265282 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:29.265301 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:29.265656 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:29.266040 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:29.266042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:29.266044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:29.266188 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:29.266704 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:29.266709 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:29.267062 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.267268 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.267575 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260807 06:16:29.270216 5601 heartbeater.cc:347] Connected to a master server at 127.9.133.190:33757
I20260807 06:16:29.270515 5601 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:29.271173 5601 heartbeater.cc:511] Master 127.9.133.190:33757 requested a full tablet report, sending...
I20260807 06:16:29.273233 5314 ts_manager.cc:194] Registered new tserver with Master: 89392ca71bad4e3d8508306b6293d4cb (127.9.133.129:42201)
WARNING: no policy specified for kudu/127.9.133.130@KRBTEST.COM; defaulting to no policy
I20260807 06:16:29.275171 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.129@KRBTEST.COM'} at 127.9.133.129:37929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
Entry for principal kudu/127.9.133.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab.
I20260807 06:16:29.294251 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:33757
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:29.424559 5606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:29.424875 5606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:29.424918 5606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:29.428932 5606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:29.428984 5606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:29.429040 5606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:29.434440 5606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.130.keytab
--principal=kudu/127.9.133.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:29.435796 5606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:29.437049 5606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:29.448784 5613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:29.448877 5614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:29.449048 5606 server_base.cc:1034] running on GCE node
W20260807 06:16:29.448784 5617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:29.449625 5606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:29.450335 5606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:29.451504 5606 hybrid_clock.cc:648] HybridClock initialized: now 1786083389451477 us; error 34 us; skew 500 ppm
Aug 07 06:16:29 dist-test-slave-h3ft krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083389, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:29.454667 5606 init.cc:377] Logged in from keytab as kudu/127.9.133.130@KRBTEST.COM (short username kudu)
I20260807 06:16:29.455760 5606 webserver.cc:460] Webserver started at http://127.9.133.130:38553/ using document root <none> and password file <none>
I20260807 06:16:29.456445 5606 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:29.456516 5606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:29.456771 5606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:29.459311 5606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "990fc3bdd3784c098a510f87f886a515"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "d2512f3d3d5e82852efc27d3e04686c6"
server_key_iv: "b619d23e03674ab1f612794eb583c00f"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.459972 5606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "990fc3bdd3784c098a510f87f886a515"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "d2512f3d3d5e82852efc27d3e04686c6"
server_key_iv: "b619d23e03674ab1f612794eb583c00f"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.464498 5606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:29.467430 5626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.468959 5606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:16:29.469144 5606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "990fc3bdd3784c098a510f87f886a515"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "d2512f3d3d5e82852efc27d3e04686c6"
server_key_iv: "b619d23e03674ab1f612794eb583c00f"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.469287 5606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:29.488252 5606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:29.491926 5606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:29.492187 5606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:29.492976 5606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:29.494019 5606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:29.494094 5606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.494176 5606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:29.494221 5606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.505162 5606 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:33339
I20260807 06:16:29.505182 5780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:33339 every 8 connection(s)
I20260807 06:16:29.506318 5606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:29.513111 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 5606
I20260807 06:16:29.513314 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:16:29.513773 9750 external_mini_cluster.cc:1428] Setting key f87b05171774a8af04d60df9ca6cacec
Aug 07 06:16:29 dist-test-slave-h3ft krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083389, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.130@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:29.525089 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.508720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:37937 (local address 127.9.133.190:33757)
0807 06:16:29.508934 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.508939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.509938 (+ 999us) server_negotiation.cc:407] Connection header received
0807 06:16:29.511954 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.511957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.512019 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.512104 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:29.514145 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.514738 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.516619 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.516863 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.516951 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.520771 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:29.520776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:29.520778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:29.520811 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:29.522543 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.523241 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.523246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.523253 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.523417 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.523799 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.523802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.523803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.524020 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:29.524136 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.524723 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.524874 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 06:16:29.526111 5783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.509147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:33757 (local address 127.9.133.130:37937)
0807 06:16:29.509727 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.509785 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.511683 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.512323 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.512347 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.512965 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:29.513941 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.513970 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.514897 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.514900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.516459 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.516478 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.517053 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.517062 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.517385 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.518251 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:29.518285 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:29.520537 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:29.522701 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:29.522708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:29.522729 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:29.523126 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:29.523516 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:29.523520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:29.523522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:29.523679 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:29.524132 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:29.524141 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:29.524451 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.524666 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.525059 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.9.133.131@KRBTEST.COM; defaulting to no policy
I20260807 06:16:29.527611 5781 heartbeater.cc:347] Connected to a master server at 127.9.133.190:33757
I20260807 06:16:29.527944 5781 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:29.528661 5781 heartbeater.cc:511] Master 127.9.133.190:33757 requested a full tablet report, sending...
I20260807 06:16:29.530068 5314 ts_manager.cc:194] Registered new tserver with Master: 990fc3bdd3784c098a510f87f886a515 (127.9.133.130:33339)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.131@KRBTEST.COM" created.
I20260807 06:16:29.530879 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.130@KRBTEST.COM'} at 127.9.133.130:37937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
Entry for principal kudu/127.9.133.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab.
I20260807 06:16:29.550647 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:33757
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:29.687469 5788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:29.687830 5788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:29.687878 5788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:29.692159 5788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:29.692232 5788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:29.692327 5788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:29.698249 5788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.131.keytab
--principal=kudu/127.9.133.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:29.699766 5788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:29.701143 5788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:29.713847 5794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:29.713907 5797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:29.713907 5795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:29.714573 5788 server_base.cc:1034] running on GCE node
I20260807 06:16:29.715170 5788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:29.716022 5788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:29.717231 5788 hybrid_clock.cc:648] HybridClock initialized: now 1786083389717198 us; error 43 us; skew 500 ppm
Aug 07 06:16:29 dist-test-slave-h3ft krb5kdc[5256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083389, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:29.725124 5788 init.cc:377] Logged in from keytab as kudu/127.9.133.131@KRBTEST.COM (short username kudu)
I20260807 06:16:29.726296 5788 webserver.cc:460] Webserver started at http://127.9.133.131:42037/ using document root <none> and password file <none>
I20260807 06:16:29.727097 5788 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:29.727200 5788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:29.727466 5788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:29.729982 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "7d252c0839b34dc68bcfb06a9813cc70"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "6f009526e927bc1b592b189cbe0b622e"
server_key_iv: "34fbd28818605ba56f48f63fe31f883e"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.730659 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "7d252c0839b34dc68bcfb06a9813cc70"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "6f009526e927bc1b592b189cbe0b622e"
server_key_iv: "34fbd28818605ba56f48f63fe31f883e"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.735414 5788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 06:16:29.738400 5808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.740217 5788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:29.740329 5788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "7d252c0839b34dc68bcfb06a9813cc70"
format_stamp: "Formatted at 2026-08-07 06:16:29 on dist-test-slave-h3ft"
server_key: "6f009526e927bc1b592b189cbe0b622e"
server_key_iv: "34fbd28818605ba56f48f63fe31f883e"
server_key_version: "encryptionkey@0"
I20260807 06:16:29.740435 5788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:29.761638 5788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:29.765537 5788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:29.765743 5788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:29.766443 5788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:29.767557 5788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:29.767619 5788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.767683 5788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:29.767719 5788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:29.778229 5788 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:37805
I20260807 06:16:29.778254 5961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:37805 every 8 connection(s)
I20260807 06:16:29.779439 5788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:29.780360 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 5788
I20260807 06:16:29.780529 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:16:29.780933 9750 external_mini_cluster.cc:1428] Setting key 452abf0cc30d9631730132b694214804
Aug 07 06:16:29 dist-test-slave-h3ft krb5kdc[5256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083389, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.131@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:29.795667 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.781878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:36191 (local address 127.9.133.190:33757)
0807 06:16:29.782043 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.782047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.783209 (+ 1162us) server_negotiation.cc:407] Connection header received
0807 06:16:29.784990 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.784994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.785061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.785148 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:29.786516 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.787215 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.788471 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.788727 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.788848 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.791746 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:29.791750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:29.791760 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:29.791792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:29.793263 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.793819 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.793822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.793824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.793919 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.794292 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.794295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.794297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.794565 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:29.794681 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.795327 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.795476 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 06:16:29.796800 5967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.782411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:33757 (local address 127.9.133.131:36191)
0807 06:16:29.782992 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.783085 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.784742 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.785352 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.785366 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.785781 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:29.786349 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.786376 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.787351 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.787363 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.788328 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.788338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.788905 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.788909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.789115 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.789762 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:16:29.789780 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:29.791557 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:29.793406 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:29.793412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:29.793430 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:29.793709 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:29.794028 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:16:29.794031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:16:29.794033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:16:29.794189 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:16:29.794687 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:29.794697 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:29.794998 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.795283 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.795683 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":140,"threads_started":1}
I20260807 06:16:29.798151 5965 heartbeater.cc:347] Connected to a master server at 127.9.133.190:33757
I20260807 06:16:29.798457 5965 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:29.799144 5965 heartbeater.cc:511] Master 127.9.133.190:33757 requested a full tablet report, sending...
I20260807 06:16:29.800359 5314 ts_manager.cc:194] Registered new tserver with Master: 7d252c0839b34dc68bcfb06a9813cc70 (127.9.133.131:37805)
I20260807 06:16:29.800992 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.133.131@KRBTEST.COM'} at 127.9.133.131:36191
I20260807 06:16:29.808010 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:16:29.819259 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.810512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.9.133.190:33757)
0807 06:16:29.810663 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.810667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.811129 (+ 462us) server_negotiation.cc:407] Connection header received
0807 06:16:29.811251 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.811254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.811306 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.811377 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:29.812219 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.812793 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.813640 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.813865 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.813975 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.815587 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:29.815590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:29.815592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:29.815619 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:29.817233 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.817678 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.817681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.817683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.817781 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:29.818139 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:29.818143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:29.818144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:29.818452 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:29.818603 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.818823 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.819027 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:16:29.822660 5314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:16:29.826578 5314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:16:29.845484 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.839167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:37805 (local address 127.0.0.1:39072)
0807 06:16:29.839742 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.839755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.839840 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.840714 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.840717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.840737 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:29.840828 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.840833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.842447 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.842453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.843383 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.843394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.844735 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.844739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.844879 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.844933 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.845077 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.845227 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":144,"threads_started":1}
I20260807 06:16:29.845798 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.838497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33339 (local address 127.0.0.1:41376)
0807 06:16:29.839505 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.839519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.839628 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.840520 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.840525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.840542 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:29.840689 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.840699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.842205 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.842208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.843820 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.843843 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.845357 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.845361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.845456 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.845475 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.845569 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.845648 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":77,"thread_start_us":154,"threads_started":1}
I20260807 06:16:29.845943 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.838963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.9.133.130:33339)
0807 06:16:29.839537 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.839555 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.839584 (+ 29us) server_negotiation.cc:407] Connection header received
0807 06:16:29.839811 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.839819 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.840152 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.840320 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:29.840832 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.842073 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.843967 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.844968 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.845103 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.845279 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.845613 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.845736 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":160,"threads_started":1}
I20260807 06:16:29.846123 5985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.839755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.9.133.131:37805)
0807 06:16:29.840211 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.840230 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.840257 (+ 27us) server_negotiation.cc:407] Connection header received
0807 06:16:29.840344 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.840348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.840547 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.840981 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:29.841183 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.842305 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.843516 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.844542 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.844706 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.844896 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.845799 (+ 903us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.845948 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":182,"threads_started":1}
I20260807 06:16:29.847595 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.838056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42201 (local address 127.0.0.1:57144)
0807 06:16:29.839115 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.839181 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.839410 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.840206 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.840213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.840239 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:29.840425 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.840435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.842068 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.842072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.844147 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.844156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.847177 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.847181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.847276 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.847301 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.847369 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.847457 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":161,"thread_start_us":105,"threads_started":1}
I20260807 06:16:29.847705 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.838470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.9.133.129:42201)
0807 06:16:29.839314 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.839323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.839357 (+ 34us) server_negotiation.cc:407] Connection header received
0807 06:16:29.839704 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.839712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.840008 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.840194 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:29.840743 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.841797 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.844500 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.847056 (+ 2556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.847190 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.847365 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.847421 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.847529 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":166,"threads_started":1}
I20260807 06:16:29.849808 5869 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf50adfce824d229757ce986c0bc07d (DEFAULT_TABLE table=test-table [id=d2058ae7d1f3497a8bf7426e6124b397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:29.849802 5686 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf50adfce824d229757ce986c0bc07d (DEFAULT_TABLE table=test-table [id=d2058ae7d1f3497a8bf7426e6124b397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:29.850524 5510 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf50adfce824d229757ce986c0bc07d (DEFAULT_TABLE table=test-table [id=d2058ae7d1f3497a8bf7426e6124b397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:29.851168 5686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf50adfce824d229757ce986c0bc07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:29.851172 5869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf50adfce824d229757ce986c0bc07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:29.851555 5510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf50adfce824d229757ce986c0bc07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:29.860133 5986 tablet_bootstrap.cc:492] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Bootstrap starting.
I20260807 06:16:29.860546 5988 tablet_bootstrap.cc:492] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: Bootstrap starting.
I20260807 06:16:29.863197 5986 tablet_bootstrap.cc:654] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:29.863197 5988 tablet_bootstrap.cc:654] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:29.863500 5989 tablet_bootstrap.cc:492] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Bootstrap starting.
I20260807 06:16:29.864449 5988 log.cc:824] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:29.864689 5986 log.cc:824] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:29.866465 5989 tablet_bootstrap.cc:654] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:29.867187 5988 tablet_bootstrap.cc:492] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: No bootstrap required, opened a new log
I20260807 06:16:29.867192 5986 tablet_bootstrap.cc:492] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: No bootstrap required, opened a new log
I20260807 06:16:29.867434 5988 ts_tablet_manager.cc:1397] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260807 06:16:29.867439 5986 ts_tablet_manager.cc:1397] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 06:16:29.867462 5989 log.cc:824] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:29.869881 5989 tablet_bootstrap.cc:492] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: No bootstrap required, opened a new log
I20260807 06:16:29.870103 5989 ts_tablet_manager.cc:1397] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 06:16:29.871613 5988 raft_consensus.cc:348] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.871686 5986 raft_consensus.cc:348] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.871903 5988 raft_consensus.cc:374] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:29.871968 5988 raft_consensus.cc:729] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89392ca71bad4e3d8508306b6293d4cb, State: Initialized, Role: FOLLOWER
I20260807 06:16:29.872006 5986 raft_consensus.cc:374] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:29.872087 5986 raft_consensus.cc:729] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d252c0839b34dc68bcfb06a9813cc70, State: Initialized, Role: FOLLOWER
I20260807 06:16:29.872550 5988 consensus_queue.cc:260] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.872603 5986 consensus_queue.cc:260] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.873656 5988 ts_tablet_manager.cc:1428] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 06:16:29.873754 5986 ts_tablet_manager.cc:1428] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:16:29.874135 5965 heartbeater.cc:503] Master 127.9.133.190:33757 was elected leader, sending a full tablet report...
I20260807 06:16:29.874136 5601 heartbeater.cc:503] Master 127.9.133.190:33757 was elected leader, sending a full tablet report...
I20260807 06:16:29.874217 5989 raft_consensus.cc:348] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.874487 5989 raft_consensus.cc:374] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:29.874547 5989 raft_consensus.cc:729] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990fc3bdd3784c098a510f87f886a515, State: Initialized, Role: FOLLOWER
I20260807 06:16:29.874949 5989 consensus_queue.cc:260] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.875913 5989 ts_tablet_manager.cc:1428] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 06:16:29.876287 5781 heartbeater.cc:503] Master 127.9.133.190:33757 was elected leader, sending a full tablet report...
I20260807 06:16:29.993119 5997 raft_consensus.cc:482] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:29.993445 5997 raft_consensus.cc:504] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:29.994944 5997 leader_election.cc:288] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d252c0839b34dc68bcfb06a9813cc70 (127.9.133.131:37805), 990fc3bdd3784c098a510f87f886a515 (127.9.133.130:33339)
I20260807 06:16:29.999652 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.995544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:59345 (local address 127.9.133.130:33339)
0807 06:16:29.995706 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.995712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.995996 (+ 284us) server_negotiation.cc:407] Connection header received
0807 06:16:29.996164 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.996168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.996247 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.996339 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:29.996659 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.997232 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.998440 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.998882 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.999054 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.999128 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.999383 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.999476 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 06:16:29.999652 5985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.995568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:52779 (local address 127.9.133.131:37805)
0807 06:16:29.995713 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:29.995719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:29.996263 (+ 544us) server_negotiation.cc:407] Connection header received
0807 06:16:29.996342 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:29.996345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:29.996399 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:29.996503 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:29.996791 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.997308 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.998454 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.998895 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.999063 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.999130 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:29.999385 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 06:16:29.999482 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 06:16:29.999676 6001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.995317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:37805 (local address 127.9.133.129:52779)
0807 06:16:29.995994 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.996010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.996156 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.996510 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.996513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.996530 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:29.996661 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.996667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.997419 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.997425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.998311 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.998321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.999050 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.999054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.999242 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.999272 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.999364 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.999479 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":257,"thread_start_us":118,"threads_started":1}
I20260807 06:16:29.999679 6000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:29.995317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:33339 (local address 127.9.133.129:59345)
0807 06:16:29.995875 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:29.995889 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:29.996037 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:29.996358 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:29.996360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:29.996373 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:29.996509 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.996515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.997357 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.997362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.998302 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:29.998313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:29.999054 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:29.999058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:29.999240 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:29.999270 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 06:16:29.999364 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 06:16:29.999477 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":54,"thread_start_us":175,"threads_started":1}
I20260807 06:16:30.000455 5900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf50adfce824d229757ce986c0bc07d" candidate_uuid: "89392ca71bad4e3d8508306b6293d4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d252c0839b34dc68bcfb06a9813cc70" is_pre_election: true
I20260807 06:16:30.000653 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf50adfce824d229757ce986c0bc07d" candidate_uuid: "89392ca71bad4e3d8508306b6293d4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990fc3bdd3784c098a510f87f886a515" is_pre_election: true
I20260807 06:16:30.000944 5900 raft_consensus.cc:2454] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89392ca71bad4e3d8508306b6293d4cb in term 0.
I20260807 06:16:30.001024 5713 raft_consensus.cc:2454] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89392ca71bad4e3d8508306b6293d4cb in term 0.
I20260807 06:16:30.001617 5435 leader_election.cc:302] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d252c0839b34dc68bcfb06a9813cc70, 89392ca71bad4e3d8508306b6293d4cb; no voters:
I20260807 06:16:30.002027 5997 raft_consensus.cc:2781] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:16:30.002149 5997 raft_consensus.cc:482] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:16:30.002203 5997 raft_consensus.cc:3037] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 0 FOLLOWER]: Advancing to term 1
W20260807 06:16:30.002862 5602 tablet.cc:2369] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:16:30.003962 5997 raft_consensus.cc:504] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:30.004422 5997 leader_election.cc:288] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [CANDIDATE]: Term 1 election: Requested vote from peers 7d252c0839b34dc68bcfb06a9813cc70 (127.9.133.131:37805), 990fc3bdd3784c098a510f87f886a515 (127.9.133.130:33339)
I20260807 06:16:30.005004 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf50adfce824d229757ce986c0bc07d" candidate_uuid: "89392ca71bad4e3d8508306b6293d4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990fc3bdd3784c098a510f87f886a515"
I20260807 06:16:30.005053 5900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf50adfce824d229757ce986c0bc07d" candidate_uuid: "89392ca71bad4e3d8508306b6293d4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d252c0839b34dc68bcfb06a9813cc70"
I20260807 06:16:30.005189 5713 raft_consensus.cc:3037] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:30.005254 5900 raft_consensus.cc:3037] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:30.006639 5900 raft_consensus.cc:2454] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89392ca71bad4e3d8508306b6293d4cb in term 1.
I20260807 06:16:30.006640 5713 raft_consensus.cc:2454] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89392ca71bad4e3d8508306b6293d4cb in term 1.
I20260807 06:16:30.007143 5435 leader_election.cc:302] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7d252c0839b34dc68bcfb06a9813cc70, 89392ca71bad4e3d8508306b6293d4cb, 990fc3bdd3784c098a510f87f886a515; no voters:
I20260807 06:16:30.007432 5997 raft_consensus.cc:2781] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 1 FOLLOWER]: Leader election won for term 1
W20260807 06:16:30.007716 5782 tablet.cc:2369] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:16:30.007812 5997 raft_consensus.cc:686] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [term 1 LEADER]: Becoming Leader. State: Replica: 89392ca71bad4e3d8508306b6293d4cb, State: Running, Role: LEADER
I20260807 06:16:30.008291 5997 consensus_queue.cc:237] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } }
I20260807 06:16:30.012744 5314 catalog_manager.cc:5495] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 89392ca71bad4e3d8508306b6293d4cb (127.9.133.129). New cstate: current_term: 1 leader_uuid: "89392ca71bad4e3d8508306b6293d4cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89392ca71bad4e3d8508306b6293d4cb" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 } health_report { overall_health: UNKNOWN } } }
W20260807 06:16:30.030673 5966 tablet.cc:2369] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:16:30.037799 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.033549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.9.133.129:42201)
0807 06:16:30.033736 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:30.033742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:30.033759 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:16:30.033877 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:30.033880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:30.033956 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:30.034092 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:16:30.034429 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.035078 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.036021 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.036253 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.036364 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:30.036742 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:16:30.037339 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:16:30.037499 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:30.037534 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 06:16:30.037605 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 06:16:30.046211 5713 raft_consensus.cc:1261] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 1 FOLLOWER]: Refusing update from remote peer 89392ca71bad4e3d8508306b6293d4cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:30.046211 5900 raft_consensus.cc:1261] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 1 FOLLOWER]: Refusing update from remote peer 89392ca71bad4e3d8508306b6293d4cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:30.047255 5997 consensus_queue.cc:1035] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d252c0839b34dc68bcfb06a9813cc70" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 37805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:30.047569 6005 consensus_queue.cc:1035] T 6bf50adfce824d229757ce986c0bc07d P 89392ca71bad4e3d8508306b6293d4cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "990fc3bdd3784c098a510f87f886a515" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 33339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:30.059381 6009 mvcc.cc:205] Tried to move back new op lower bound from 7315797565618290688 to 7315797565477859328. Current Snapshot: MvccSnapshot[applied={T|T < 7315797565618290688}]
I20260807 06:16:30.070047 5312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:30.070355 5312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:30.074182 5312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e56f6a1b079b410984c9be872cfef9a9: Sending DeleteTablet for 3 replicas of tablet 6bf50adfce824d229757ce986c0bc07d
I20260807 06:16:30.075489 5510 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf50adfce824d229757ce986c0bc07d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:30 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:57144
I20260807 06:16:30.075608 5869 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf50adfce824d229757ce986c0bc07d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:30 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:39072
I20260807 06:16:30.075598 5686 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf50adfce824d229757ce986c0bc07d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:30 UTC) from {username='kudu', principal='kudu/127.9.133.190@KRBTEST.COM'} at 127.0.0.1:41376
I20260807 06:16:30.076411 6023 tablet_replica.cc:330] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: stopping tablet replica
I20260807 06:16:30.077026 6023 raft_consensus.cc:2229] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:30.077529 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 5400
I20260807 06:16:30.077566 6023 raft_consensus.cc:2258] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:30.083796 6024 tablet_replica.cc:330] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: stopping tablet replica
I20260807 06:16:30.085223 6023 ts_tablet_manager.cc:1905] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:30.086102 6024 raft_consensus.cc:2229] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:30.086647 6024 raft_consensus.cc:2258] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:30.088884 6024 ts_tablet_manager.cc:1905] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:30.090262 6023 ts_tablet_manager.cc:1918] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:30.090399 6023 log.cc:1196] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/6bf50adfce824d229757ce986c0bc07d
I20260807 06:16:30.090847 6023 ts_tablet_manager.cc:1939] T 6bf50adfce824d229757ce986c0bc07d P 7d252c0839b34dc68bcfb06a9813cc70: Deleting consensus metadata
I20260807 06:16:30.091806 6024 ts_tablet_manager.cc:1918] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:30.091946 6024 log.cc:1196] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/6bf50adfce824d229757ce986c0bc07d
I20260807 06:16:30.092324 6024 ts_tablet_manager.cc:1939] T 6bf50adfce824d229757ce986c0bc07d P 990fc3bdd3784c098a510f87f886a515: Deleting consensus metadata
I20260807 06:16:30.092760 5298 catalog_manager.cc:4841] TS 7d252c0839b34dc68bcfb06a9813cc70 (127.9.133.131:37805): tablet 6bf50adfce824d229757ce986c0bc07d (table test-table [id=d2058ae7d1f3497a8bf7426e6124b397]) successfully deleted
I20260807 06:16:30.093727 5296 catalog_manager.cc:4841] TS 990fc3bdd3784c098a510f87f886a515 (127.9.133.130:33339): tablet 6bf50adfce824d229757ce986c0bc07d (table test-table [id=d2058ae7d1f3497a8bf7426e6124b397]) successfully deleted
W20260807 06:16:30.097313 5299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:42201 (error 108)
I20260807 06:16:30.097575 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 5606
I20260807 06:16:30.104015 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.103459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:42201 (local address 127.0.0.1:57152)
0807 06:16:30.103754 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:30.103858 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:42201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260807 06:16:30.104297 5299 catalog_manager.cc:4584] TS 89392ca71bad4e3d8508306b6293d4cb (127.9.133.129:42201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6bf50adfce824d229757ce986c0bc07d: Network error: Client connection negotiation failed: client connection to 127.9.133.129:42201: connect: Connection refused (error 111)
I20260807 06:16:30.117223 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 5788
I20260807 06:16:30.135504 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 5276
2026-08-07T06:16:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2905 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T06:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:30Z Disabled control of system clock
I20260807 06:16:30.182365 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45339
--webserver_interface=127.9.133.190
--webserver_port=0
--builtin_ntp_servers=127.9.133.148:34607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:16:30.317521 6030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:30.317903 6030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:30.317946 6030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:30.322046 6030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:30.322141 6030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:30.322160 6030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:30.322207 6030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:30.322227 6030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:30.327279 6030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:45339
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:30.328759 6030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:30.330699 6030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:30.341806 6042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:30.341871 6049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:30.341867 6039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:30.342023 6030 server_base.cc:1034] running on GCE node
I20260807 06:16:30.343372 6030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:30.344921 6030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:30.346148 6030 hybrid_clock.cc:648] HybridClock initialized: now 1786083390346125 us; error 36 us; skew 500 ppm
I20260807 06:16:30.348804 6030 webserver.cc:460] Webserver started at http://127.9.133.190:35833/ using document root <none> and password file <none>
I20260807 06:16:30.349606 6030 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:30.349711 6030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:30.350019 6030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:30.352905 6030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "77e6959ab50042938245ea6257792266"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "ef796b93aa59770f2e6a15153882b231"
server_key_iv: "9e84dbd97ca963603f94fb238be7e54f"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.353744 6030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "77e6959ab50042938245ea6257792266"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "ef796b93aa59770f2e6a15153882b231"
server_key_iv: "9e84dbd97ca963603f94fb238be7e54f"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.358605 6030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 06:16:30.361783 6056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.363447 6030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:30.363605 6030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "77e6959ab50042938245ea6257792266"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "ef796b93aa59770f2e6a15153882b231"
server_key_iv: "9e84dbd97ca963603f94fb238be7e54f"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.363741 6030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:30.391490 6030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:30.392572 6030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:30.392855 6030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:30.401949 6030 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:45339
I20260807 06:16:30.401970 6126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:45339 every 8 connection(s)
I20260807 06:16:30.403255 6030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:30.407289 6127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:30.410604 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 6030
I20260807 06:16:30.410763 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:30.411207 9750 external_mini_cluster.cc:1428] Setting key c55341b980735d2504403f3f12a8981b
I20260807 06:16:30.414947 6127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266: Bootstrap starting.
I20260807 06:16:30.417712 6127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:30.418637 6127 log.cc:824] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:30.420878 6127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266: No bootstrap required, opened a new log
I20260807 06:16:30.422871 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.413126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.9.133.190:45339)
0807 06:16:30.413926 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:30.413946 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:30.413992 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:16:30.414746 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:30.414767 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:30.415393 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:30.415853 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:30.416399 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.417556 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.418669 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.419367 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.419578 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:30.420674 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:30.420680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:30.420696 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:30.421256 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:30.421327 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:30.421352 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:30.421498 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:30.421646 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 06:16:30.421976 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":146,"threads_started":1}
I20260807 06:16:30.425372 6127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e6959ab50042938245ea6257792266" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45339 } }
I20260807 06:16:30.425720 6127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:30.425802 6127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e6959ab50042938245ea6257792266, State: Initialized, Role: FOLLOWER
I20260807 06:16:30.426295 6127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e6959ab50042938245ea6257792266" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45339 } }
I20260807 06:16:30.426458 6127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:30.426556 6127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:30.426687 6127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:30.427816 6127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e6959ab50042938245ea6257792266" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45339 } }
I20260807 06:16:30.428212 6127 leader_election.cc:302] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77e6959ab50042938245ea6257792266; no voters:
I20260807 06:16:30.428601 6127 leader_election.cc:288] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:30.428736 6132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:30.429044 6132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [term 1 LEADER]: Becoming Leader. State: Replica: 77e6959ab50042938245ea6257792266, State: Running, Role: LEADER
I20260807 06:16:30.429466 6132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e6959ab50042938245ea6257792266" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45339 } }
I20260807 06:16:30.430079 6127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:30.431586 6137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e6959ab50042938245ea6257792266. Latest consensus state: current_term: 1 leader_uuid: "77e6959ab50042938245ea6257792266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e6959ab50042938245ea6257792266" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45339 } } }
I20260807 06:16:30.431741 6137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:30.432214 6133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77e6959ab50042938245ea6257792266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e6959ab50042938245ea6257792266" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45339 } } }
I20260807 06:16:30.432415 6133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:30.432870 6149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:30.436642 6149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:30.444060 6149 catalog_manager.cc:1347] Generated new cluster ID: 2edb53bdc988422897e0b1ea499fba7c
I20260807 06:16:30.444178 6149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:30.460219 6149 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:30.461571 6149 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:30.480139 6149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266: Generated new TSK 0
I20260807 06:16:30.481206 6149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:30.491699 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.129:0
--local_ip_for_outbound_sockets=127.9.133.129
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45339
--builtin_ntp_servers=127.9.133.148:34607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 06:16:30.634104 6159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:30.634457 6159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:30.634533 6159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:30.638744 6159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:30.638818 6159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:30.638877 6159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.129
I20260807 06:16:30.643970 6159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.133.129
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.129
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:30.645469 6159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:30.647295 6159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:30.658792 6166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:30.658776 6172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:30.658775 6168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:30.659157 6159 server_base.cc:1034] running on GCE node
I20260807 06:16:30.659683 6159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:30.660504 6159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:30.661753 6159 hybrid_clock.cc:648] HybridClock initialized: now 1786083390661738 us; error 32 us; skew 500 ppm
I20260807 06:16:30.664083 6159 webserver.cc:460] Webserver started at http://127.9.133.129:36005/ using document root <none> and password file <none>
I20260807 06:16:30.664793 6159 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:30.664896 6159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:30.665179 6159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:30.667698 6159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data/instance:
uuid: "8f2c44e2cd9a4470977a3eab2abd77a7"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "2e28eb00846f0096387d3e31addbe00d"
server_key_iv: "8c7ec62ead8a4df86b5d826ece66cdaf"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.668452 6159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance:
uuid: "8f2c44e2cd9a4470977a3eab2abd77a7"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "2e28eb00846f0096387d3e31addbe00d"
server_key_iv: "8c7ec62ead8a4df86b5d826ece66cdaf"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.672974 6159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:30.675985 6183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.677378 6159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:30.677531 6159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal
uuid: "8f2c44e2cd9a4470977a3eab2abd77a7"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "2e28eb00846f0096387d3e31addbe00d"
server_key_iv: "8c7ec62ead8a4df86b5d826ece66cdaf"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.677663 6159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:30.695731 6159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:30.696664 6159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:30.696907 6159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:30.697664 6159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:30.698907 6159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:30.698988 6159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.699139 6159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:30.699186 6159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.710314 6159 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.129:39527
I20260807 06:16:30.710347 6332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.129:39527 every 8 connection(s)
I20260807 06:16:30.711501 6159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/data/info.pb
I20260807 06:16:30.721946 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 6159
I20260807 06:16:30.722116 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-0/wal/instance
I20260807 06:16:30.722579 9750 external_mini_cluster.cc:1428] Setting key 0402c12aae452abc1257141b87f1ca27
I20260807 06:16:30.722806 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.713871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:44357 (local address 127.9.133.190:45339)
0807 06:16:30.714055 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:30.714059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:30.715392 (+ 1333us) server_negotiation.cc:407] Connection header received
0807 06:16:30.717063 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:30.717068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:30.717138 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:30.717261 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:30.718322 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.718922 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.720227 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.720486 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.720608 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:30.722020 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:30.722027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:30.722030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:30.722049 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:30.722081 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:30.722093 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:30.722165 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:30.722398 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 06:16:30.722544 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 06:16:30.724009 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.714366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45339 (local address 127.9.133.129:44357)
0807 06:16:30.715210 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:30.715259 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:30.716805 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:30.717474 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:30.717482 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:30.717578 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:30.718160 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:30.718173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.719126 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.719129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:30.720088 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:30.720098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.720664 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.720668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:30.720863 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:30.721591 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:30.721593 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:30.721658 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:30.721758 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:30.722207 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:30.722219 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:30.722225 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 06:16:30.722366 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 06:16:30.722747 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":147,"threads_started":1}
I20260807 06:16:30.725448 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.130:0
--local_ip_for_outbound_sockets=127.9.133.130
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45339
--builtin_ntp_servers=127.9.133.148:34607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:16:30.725586 6333 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45339
I20260807 06:16:30.725878 6333 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:30.726555 6333 heartbeater.cc:511] Master 127.9.133.190:45339 requested a full tablet report, sending...
I20260807 06:16:30.728600 6081 ts_manager.cc:194] Registered new tserver with Master: 8f2c44e2cd9a4470977a3eab2abd77a7 (127.9.133.129:39527)
I20260807 06:16:30.730162 6081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.129:44357
W20260807 06:16:30.871853 6336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:30.872216 6336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:30.872260 6336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:30.876497 6336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:30.876575 6336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:30.876637 6336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.130
I20260807 06:16:30.881760 6336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.133.130
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.130
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:30.883340 6336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:30.885087 6336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:30.896750 6343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:30.896726 6344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:30.896720 6346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:30.897298 6336 server_base.cc:1034] running on GCE node
I20260807 06:16:30.897742 6336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:30.898451 6336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:30.899643 6336 hybrid_clock.cc:648] HybridClock initialized: now 1786083390899625 us; error 27 us; skew 500 ppm
I20260807 06:16:30.902060 6336 webserver.cc:460] Webserver started at http://127.9.133.130:46161/ using document root <none> and password file <none>
I20260807 06:16:30.902832 6336 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:30.902907 6336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:30.903344 6336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:30.905977 6336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data/instance:
uuid: "c5d7050ebdbd4ec29b326dc024bc27ba"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "7dad5bfad5b42dbb70d880fb3dc00eb3"
server_key_iv: "fea9f2c1291b9c3b3a70bf221097e459"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.906653 6336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance:
uuid: "c5d7050ebdbd4ec29b326dc024bc27ba"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "7dad5bfad5b42dbb70d880fb3dc00eb3"
server_key_iv: "fea9f2c1291b9c3b3a70bf221097e459"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.911332 6336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 06:16:30.914610 6352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.916334 6336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:30.916493 6336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal
uuid: "c5d7050ebdbd4ec29b326dc024bc27ba"
format_stamp: "Formatted at 2026-08-07 06:16:30 on dist-test-slave-h3ft"
server_key: "7dad5bfad5b42dbb70d880fb3dc00eb3"
server_key_iv: "fea9f2c1291b9c3b3a70bf221097e459"
server_key_version: "encryptionkey@0"
I20260807 06:16:30.916646 6336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:30.944149 6336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:30.945170 6336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:30.945438 6336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:30.946307 6336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:30.947736 6336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:30.947824 6336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.947918 6336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:30.947988 6336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:30.959326 6336 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.130:43365
I20260807 06:16:30.959367 6516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.130:43365 every 8 connection(s)
I20260807 06:16:30.960520 6336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/data/info.pb
I20260807 06:16:30.967058 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 6336
I20260807 06:16:30.967206 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal/instance
I20260807 06:16:30.967648 9750 external_mini_cluster.cc:1428] Setting key 578771d0ff9e07915af2aad117ea2499
I20260807 06:16:30.970556 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.133.131:0
--local_ip_for_outbound_sockets=127.9.133.131
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45339
--builtin_ntp_servers=127.9.133.148:34607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 06:16:30.973513 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.962953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.130:56677 (local address 127.9.133.190:45339)
0807 06:16:30.963154 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:30.963159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:30.964518 (+ 1359us) server_negotiation.cc:407] Connection header received
0807 06:16:30.966212 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:30.966215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:30.966278 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:30.966362 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:30.967902 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.968722 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.970629 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.970956 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.971136 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:30.972777 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:30.972781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:30.972783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:30.972796 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:30.972823 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:30.972832 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:30.972899 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:30.973167 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 06:16:30.973262 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 06:16:30.974609 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:30.963484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45339 (local address 127.9.133.130:56677)
0807 06:16:30.964330 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:30.964382 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:30.965964 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:30.966567 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:30.966575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:30.966725 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:30.967711 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:30.967737 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.968879 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.968883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:30.970477 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:30.970491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:30.971201 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:30.971208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:30.971485 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:30.972370 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:30.972375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:30.972450 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:30.972558 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:30.972936 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:30.972950 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:30.972960 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 06:16:30.973123 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 06:16:30.973520 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":129,"threads_started":1}
I20260807 06:16:30.976084 6517 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45339
I20260807 06:16:30.976380 6517 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:30.977111 6517 heartbeater.cc:511] Master 127.9.133.190:45339 requested a full tablet report, sending...
I20260807 06:16:30.978430 6081 ts_manager.cc:194] Registered new tserver with Master: c5d7050ebdbd4ec29b326dc024bc27ba (127.9.133.130:43365)
I20260807 06:16:30.979225 6081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.130:56677
W20260807 06:16:31.113193 6522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:31.113572 6522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:31.113615 6522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:31.118033 6522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:31.118114 6522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:31.118178 6522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.133.131
I20260807 06:16:31.123562 6522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:34607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.133.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.133.131
--webserver_port=0
--tserver_master_addrs=127.9.133.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.133.131
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:31.125115 6522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:31.126994 6522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:31.139182 6531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:31.139135 6529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:31.139271 6522 server_base.cc:1034] running on GCE node
W20260807 06:16:31.139570 6528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:31.140057 6522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:31.140851 6522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:31.142056 6522 hybrid_clock.cc:648] HybridClock initialized: now 1786083391142024 us; error 53 us; skew 500 ppm
I20260807 06:16:31.144366 6522 webserver.cc:460] Webserver started at http://127.9.133.131:33521/ using document root <none> and password file <none>
I20260807 06:16:31.145121 6522 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:31.145223 6522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:31.145496 6522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:31.148146 6522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data/instance:
uuid: "1365b9f4e7584e5e9b7ce7968d2b4355"
format_stamp: "Formatted at 2026-08-07 06:16:31 on dist-test-slave-h3ft"
server_key: "4c157839186e41fbb946ce96ad0e9b11"
server_key_iv: "f9d7e00bdf3371b3d66bb867a5b47e50"
server_key_version: "encryptionkey@0"
I20260807 06:16:31.148824 6522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance:
uuid: "1365b9f4e7584e5e9b7ce7968d2b4355"
format_stamp: "Formatted at 2026-08-07 06:16:31 on dist-test-slave-h3ft"
server_key: "4c157839186e41fbb946ce96ad0e9b11"
server_key_iv: "f9d7e00bdf3371b3d66bb867a5b47e50"
server_key_version: "encryptionkey@0"
I20260807 06:16:31.153538 6522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:31.156569 6537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:31.158099 6522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:16:31.158249 6522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal
uuid: "1365b9f4e7584e5e9b7ce7968d2b4355"
format_stamp: "Formatted at 2026-08-07 06:16:31 on dist-test-slave-h3ft"
server_key: "4c157839186e41fbb946ce96ad0e9b11"
server_key_iv: "f9d7e00bdf3371b3d66bb867a5b47e50"
server_key_version: "encryptionkey@0"
I20260807 06:16:31.158385 6522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:31.172683 6522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:31.173537 6522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:31.173756 6522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:31.174463 6522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 06:16:31.175711 6522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 06:16:31.175786 6522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:31.175872 6522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 06:16:31.175920 6522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:31.186370 6522 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.131:34439
I20260807 06:16:31.186401 6678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.131:34439 every 8 connection(s)
I20260807 06:16:31.187635 6522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/data/info.pb
I20260807 06:16:31.191624 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 6522
I20260807 06:16:31.191798 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal/instance
I20260807 06:16:31.192222 9750 external_mini_cluster.cc:1428] Setting key 663f521332446bd1936ce4bc8724b13b
I20260807 06:16:31.199249 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.190105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.131:41569 (local address 127.9.133.190:45339)
0807 06:16:31.190289 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.190294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.191828 (+ 1534us) server_negotiation.cc:407] Connection header received
0807 06:16:31.193850 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.193854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.193927 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.194045 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:31.195155 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.195747 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.197009 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.197253 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.197371 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.198655 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:31.198658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:31.198661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:31.198674 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:31.198698 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:31.198707 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:31.198787 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.198955 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.199069 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:16:31.200327 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.190642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:45339 (local address 127.9.133.131:41569)
0807 06:16:31.191620 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.191673 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:31.193606 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:31.194255 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:31.194264 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:31.194366 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:16:31.194959 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.194972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.195884 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.195895 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.196861 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.196870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.197423 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.197427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.197622 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.198336 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 06:16:31.198337 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:16:31.198388 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 06:16:31.198463 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:16:31.198789 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:16:31.198802 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:16:31.198809 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 06:16:31.198957 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 06:16:31.199330 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":405,"threads_started":1}
I20260807 06:16:31.201658 6680 heartbeater.cc:347] Connected to a master server at 127.9.133.190:45339
I20260807 06:16:31.201992 6680 heartbeater.cc:464] Registering TS with master...
I20260807 06:16:31.202651 6680 heartbeater.cc:511] Master 127.9.133.190:45339 requested a full tablet report, sending...
I20260807 06:16:31.203915 6081 ts_manager.cc:194] Registered new tserver with Master: 1365b9f4e7584e5e9b7ce7968d2b4355 (127.9.133.131:34439)
I20260807 06:16:31.204614 6081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.133.131:41569
I20260807 06:16:31.206993 9750 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 06:16:31.213531 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.209453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.9.133.190:45339)
0807 06:16:31.209633 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.209637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.209901 (+ 264us) server_negotiation.cc:407] Connection header received
0807 06:16:31.209990 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.209993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.210040 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.210118 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:31.210413 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.210961 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.211886 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.212106 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.212193 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.213105 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:31.213108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:31.213110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:31.213124 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:31.213147 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:31.213156 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:31.213217 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.213318 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.213377 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 06:16:31.216885 6081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 06:16:31.220650 6081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 06:16:31.241422 6696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.233314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.9.133.131:34439)
0807 06:16:31.233792 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.233809 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.235480 (+ 1671us) server_negotiation.cc:407] Connection header received
0807 06:16:31.235609 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.235616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.235865 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.236033 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:31.236630 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.237855 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.239674 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.240583 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.240698 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.240956 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.241055 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.241173 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":180,"threads_started":1}
I20260807 06:16:31.241605 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.232564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:34439 (local address 127.0.0.1:48112)
0807 06:16:31.235343 (+ 2779us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.235362 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:31.235487 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:31.236244 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:31.236251 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:31.236277 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:31.236446 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.236453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.237993 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.237997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.239523 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.239535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.240724 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.240727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.240848 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.240888 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 06:16:31.241023 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 06:16:31.241428 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2712,"thread_start_us":154,"threads_started":1}
I20260807 06:16:31.245241 6694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.232361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39527 (local address 127.0.0.1:34858)
0807 06:16:31.234590 (+ 2229us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.234655 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:31.234826 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:31.241126 (+ 6300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:31.241129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:31.241146 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:31.241233 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.241239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.242515 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.242518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.243675 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.243686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.244793 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.244797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.244890 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.244914 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:16:31.244980 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 06:16:31.245072 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1997,"mutex_wait_us":25,"thread_start_us":137,"threads_started":1}
I20260807 06:16:31.245375 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.240292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34858 (local address 127.9.133.129:39527)
0807 06:16:31.240669 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.240676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.240704 (+ 28us) server_negotiation.cc:407] Connection header received
0807 06:16:31.240792 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.240797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.240985 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.241151 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:31.241336 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.242387 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.243817 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.244630 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.244794 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.245032 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.245086 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.245197 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":142,"threads_started":1}
I20260807 06:16:31.245956 6598 tablet_service.cc:1467] Processing CreateTablet for tablet a0d0cc646a39424f9e9d9a58ba6796db (DEFAULT_TABLE table=test-table [id=1d473cbc7cca41bf870fb2cc995f892f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:31.247392 6693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.234271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43365 (local address 127.0.0.1:39274)
0807 06:16:31.238873 (+ 4602us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.238889 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:31.239101 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:31.239875 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:31.239877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:31.239895 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:31.240014 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.240019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.241442 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.241446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.244849 (+ 3403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.244861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.246947 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.246951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.247079 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.247103 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 06:16:31.247172 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 06:16:31.247245 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4529,"mutex_wait_us":1,"thread_start_us":887,"threads_started":1}
I20260807 06:16:31.247457 6598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d0cc646a39424f9e9d9a58ba6796db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:31.247532 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.234741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.9.133.130:43365)
0807 06:16:31.235179 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.235194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.238993 (+ 3799us) server_negotiation.cc:407] Connection header received
0807 06:16:31.239273 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.239280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.239518 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.239665 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:31.240152 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.241298 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.245924 (+ 4626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.246807 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.246935 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.247170 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.247224 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.247328 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":165,"threads_started":1}
I20260807 06:16:31.249006 6245 tablet_service.cc:1467] Processing CreateTablet for tablet a0d0cc646a39424f9e9d9a58ba6796db (DEFAULT_TABLE table=test-table [id=1d473cbc7cca41bf870fb2cc995f892f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:31.250258 6245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d0cc646a39424f9e9d9a58ba6796db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:31.250294 6422 tablet_service.cc:1467] Processing CreateTablet for tablet a0d0cc646a39424f9e9d9a58ba6796db (DEFAULT_TABLE table=test-table [id=1d473cbc7cca41bf870fb2cc995f892f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 06:16:31.251487 6422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d0cc646a39424f9e9d9a58ba6796db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:31.256937 6704 tablet_bootstrap.cc:492] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Bootstrap starting.
I20260807 06:16:31.258308 6705 tablet_bootstrap.cc:492] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: Bootstrap starting.
I20260807 06:16:31.259428 6706 tablet_bootstrap.cc:492] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Bootstrap starting.
I20260807 06:16:31.259944 6704 tablet_bootstrap.cc:654] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:31.260777 6705 tablet_bootstrap.cc:654] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:31.260828 6704 log.cc:824] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:31.261621 6706 tablet_bootstrap.cc:654] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:31.261688 6705 log.cc:824] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:31.262534 6706 log.cc:824] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:31.263810 6704 tablet_bootstrap.cc:492] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: No bootstrap required, opened a new log
I20260807 06:16:31.263923 6705 tablet_bootstrap.cc:492] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: No bootstrap required, opened a new log
I20260807 06:16:31.264053 6704 ts_tablet_manager.cc:1397] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 06:16:31.264160 6705 ts_tablet_manager.cc:1397] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 06:16:31.264809 6706 tablet_bootstrap.cc:492] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: No bootstrap required, opened a new log
I20260807 06:16:31.264988 6706 ts_tablet_manager.cc:1397] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 06:16:31.268164 6705 raft_consensus.cc:348] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.268636 6705 raft_consensus.cc:374] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:31.268767 6705 raft_consensus.cc:729] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2c44e2cd9a4470977a3eab2abd77a7, State: Initialized, Role: FOLLOWER
I20260807 06:16:31.268965 6704 raft_consensus.cc:348] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.269352 6704 raft_consensus.cc:374] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:31.269425 6704 raft_consensus.cc:729] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1365b9f4e7584e5e9b7ce7968d2b4355, State: Initialized, Role: FOLLOWER
I20260807 06:16:31.269536 6705 consensus_queue.cc:260] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.269634 6706 raft_consensus.cc:348] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.269887 6706 raft_consensus.cc:374] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:31.269862 6704 consensus_queue.cc:260] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.269982 6706 raft_consensus.cc:729] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d7050ebdbd4ec29b326dc024bc27ba, State: Initialized, Role: FOLLOWER
I20260807 06:16:31.270462 6706 consensus_queue.cc:260] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.270744 6680 heartbeater.cc:503] Master 127.9.133.190:45339 was elected leader, sending a full tablet report...
I20260807 06:16:31.270887 6705 ts_tablet_manager.cc:1428] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 06:16:31.270888 6704 ts_tablet_manager.cc:1428] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260807 06:16:31.271299 6333 heartbeater.cc:503] Master 127.9.133.190:45339 was elected leader, sending a full tablet report...
I20260807 06:16:31.271461 6706 ts_tablet_manager.cc:1428] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 06:16:31.271854 6517 heartbeater.cc:503] Master 127.9.133.190:45339 was elected leader, sending a full tablet report...
I20260807 06:16:31.292104 6711 raft_consensus.cc:482] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 06:16:31.292555 6711 raft_consensus.cc:504] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.294068 6711 leader_election.cc:288] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1365b9f4e7584e5e9b7ce7968d2b4355 (127.9.133.131:34439), c5d7050ebdbd4ec29b326dc024bc27ba (127.9.133.130:43365)
I20260807 06:16:31.298589 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.294497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:35241 (local address 127.9.133.130:43365)
0807 06:16:31.294662 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.294668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.295153 (+ 485us) server_negotiation.cc:407] Connection header received
0807 06:16:31.295299 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.295301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.295381 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.295468 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:31.295792 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.296320 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.297539 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.297957 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.298057 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.298144 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.298338 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.298433 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:16:31.298641 6696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.294472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.133.129:48323 (local address 127.9.133.131:34439)
0807 06:16:31.294662 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.294669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.294978 (+ 309us) server_negotiation.cc:407] Connection header received
0807 06:16:31.295206 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.295212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.295289 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.295359 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 06:16:31.295771 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.296326 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.297563 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.297983 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.298141 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.298196 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.298414 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.298512 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 06:16:31.298621 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.294363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.130:43365 (local address 127.9.133.129:35241)
0807 06:16:31.294997 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.295042 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:31.295187 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:31.295495 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:31.295498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:31.295518 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:31.295647 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.295652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.296428 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.296432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.297397 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.297406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.298081 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.298087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.298184 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.298215 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 06:16:31.298289 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 06:16:31.298372 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":87,"thread_start_us":130,"threads_started":1}
I20260807 06:16:31.298621 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.294360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.131:34439 (local address 127.9.133.129:48323)
0807 06:16:31.294855 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.294869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:16:31.294987 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:16:31.295402 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:16:31.295408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:16:31.295434 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 06:16:31.295631 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.295638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.296467 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.296470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.297402 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:16:31.297411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.298149 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.298154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:16:31.298252 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.298290 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 06:16:31.298371 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 06:16:31.298437 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":43,"thread_start_us":180,"threads_started":1}
I20260807 06:16:31.299429 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d0cc646a39424f9e9d9a58ba6796db" candidate_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" is_pre_election: true
I20260807 06:16:31.299525 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d0cc646a39424f9e9d9a58ba6796db" candidate_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" is_pre_election: true
I20260807 06:16:31.299855 6452 raft_consensus.cc:2454] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f2c44e2cd9a4470977a3eab2abd77a7 in term 0.
I20260807 06:16:31.299855 6624 raft_consensus.cc:2454] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f2c44e2cd9a4470977a3eab2abd77a7 in term 0.
I20260807 06:16:31.300462 6185 leader_election.cc:302] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1365b9f4e7584e5e9b7ce7968d2b4355, 8f2c44e2cd9a4470977a3eab2abd77a7, c5d7050ebdbd4ec29b326dc024bc27ba; no voters:
I20260807 06:16:31.300833 6711 raft_consensus.cc:2781] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 06:16:31.300945 6711 raft_consensus.cc:482] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 06:16:31.301003 6711 raft_consensus.cc:3037] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:31.302444 6711 raft_consensus.cc:504] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.302793 6711 leader_election.cc:288] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [CANDIDATE]: Term 1 election: Requested vote from peers 1365b9f4e7584e5e9b7ce7968d2b4355 (127.9.133.131:34439), c5d7050ebdbd4ec29b326dc024bc27ba (127.9.133.130:43365)
I20260807 06:16:31.303231 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d0cc646a39424f9e9d9a58ba6796db" candidate_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba"
I20260807 06:16:31.303252 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d0cc646a39424f9e9d9a58ba6796db" candidate_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355"
I20260807 06:16:31.303359 6452 raft_consensus.cc:3037] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:31.303408 6624 raft_consensus.cc:3037] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:31.304656 6452 raft_consensus.cc:2454] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f2c44e2cd9a4470977a3eab2abd77a7 in term 1.
I20260807 06:16:31.304757 6624 raft_consensus.cc:2454] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f2c44e2cd9a4470977a3eab2abd77a7 in term 1.
I20260807 06:16:31.305042 6186 leader_election.cc:302] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f2c44e2cd9a4470977a3eab2abd77a7, c5d7050ebdbd4ec29b326dc024bc27ba; no voters:
I20260807 06:16:31.305270 6711 raft_consensus.cc:2781] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:31.305562 6711 raft_consensus.cc:686] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 1 LEADER]: Becoming Leader. State: Replica: 8f2c44e2cd9a4470977a3eab2abd77a7, State: Running, Role: LEADER
I20260807 06:16:31.305971 6711 consensus_queue.cc:237] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } }
I20260807 06:16:31.310235 6081 catalog_manager.cc:5495] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f2c44e2cd9a4470977a3eab2abd77a7 (127.9.133.129). New cstate: current_term: 1 leader_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f2c44e2cd9a4470977a3eab2abd77a7" member_type: VOTER last_known_addr { host: "127.9.133.129" port: 39527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 } health_report { overall_health: UNKNOWN } } }
I20260807 06:16:31.326581 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.322403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.9.133.129:39527)
0807 06:16:31.322573 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:31.322580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:31.322604 (+ 24us) server_negotiation.cc:407] Connection header received
0807 06:16:31.322843 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:31.322846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:31.322916 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:31.323073 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:16:31.323350 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.323919 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.324849 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:31.325127 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:31.325247 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:31.325592 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:16:31.326140 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:16:31.326292 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:31.326331 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 06:16:31.326392 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 06:16:31.334744 6452 raft_consensus.cc:1261] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 1 FOLLOWER]: Refusing update from remote peer 8f2c44e2cd9a4470977a3eab2abd77a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:31.334745 6624 raft_consensus.cc:1261] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 1 FOLLOWER]: Refusing update from remote peer 8f2c44e2cd9a4470977a3eab2abd77a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:16:31.335650 6711 consensus_queue.cc:1035] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5d7050ebdbd4ec29b326dc024bc27ba" member_type: VOTER last_known_addr { host: "127.9.133.130" port: 43365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:31.335894 6719 consensus_queue.cc:1035] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1365b9f4e7584e5e9b7ce7968d2b4355" member_type: VOTER last_known_addr { host: "127.9.133.131" port: 34439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:16:31.347083 6723 mvcc.cc:205] Tried to move back new op lower bound from 7315797570897321984 to 7315797570792595456. Current Snapshot: MvccSnapshot[applied={T|T < 7315797570897321984}]
I20260807 06:16:31.359187 6079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:31.359449 6079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 06:16:31.362534 6079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77e6959ab50042938245ea6257792266: Sending DeleteTablet for 3 replicas of tablet a0d0cc646a39424f9e9d9a58ba6796db
I20260807 06:16:31.363536 6245 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d0cc646a39424f9e9d9a58ba6796db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:31 UTC) from {username='slave'} at 127.0.0.1:34858
I20260807 06:16:31.363565 6598 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d0cc646a39424f9e9d9a58ba6796db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:31 UTC) from {username='slave'} at 127.0.0.1:48112
I20260807 06:16:31.364172 6739 tablet_replica.cc:330] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7: stopping tablet replica
I20260807 06:16:31.364251 6422 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d0cc646a39424f9e9d9a58ba6796db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 06:16:31 UTC) from {username='slave'} at 127.0.0.1:39274
I20260807 06:16:31.364595 6739 raft_consensus.cc:2229] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 06:16:31.364871 6740 tablet_replica.cc:330] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: stopping tablet replica
I20260807 06:16:31.365160 6739 raft_consensus.cc:2258] T a0d0cc646a39424f9e9d9a58ba6796db P 8f2c44e2cd9a4470977a3eab2abd77a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:31.365366 6740 raft_consensus.cc:2229] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:31.365602 6738 tablet_replica.cc:330] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: stopping tablet replica
I20260807 06:16:31.365808 6740 raft_consensus.cc:2258] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:31.365818 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 6159
I20260807 06:16:31.367786 6740 ts_tablet_manager.cc:1905] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:31.371397 6740 ts_tablet_manager.cc:1918] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:31.371517 6740 log.cc:1196] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-1/wal/wals/a0d0cc646a39424f9e9d9a58ba6796db
I20260807 06:16:31.371860 6740 ts_tablet_manager.cc:1939] T a0d0cc646a39424f9e9d9a58ba6796db P c5d7050ebdbd4ec29b326dc024bc27ba: Deleting consensus metadata
I20260807 06:16:31.373581 6061 catalog_manager.cc:4841] TS c5d7050ebdbd4ec29b326dc024bc27ba (127.9.133.130:43365): tablet a0d0cc646a39424f9e9d9a58ba6796db (table test-table [id=1d473cbc7cca41bf870fb2cc995f892f]) successfully deleted
I20260807 06:16:31.373651 6738 raft_consensus.cc:2229] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 06:16:31.374073 6738 raft_consensus.cc:2258] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 06:16:31.375841 6738 ts_tablet_manager.cc:1905] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 06:16:31.378804 6738 ts_tablet_manager.cc:1918] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 06:16:31.378949 6738 log.cc:1196] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Deleting WAL directory at /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786083100784614-9750-0/minicluster-data/ts-2/wal/wals/a0d0cc646a39424f9e9d9a58ba6796db
I20260807 06:16:31.379320 6738 ts_tablet_manager.cc:1939] T a0d0cc646a39424f9e9d9a58ba6796db P 1365b9f4e7584e5e9b7ce7968d2b4355: Deleting consensus metadata
I20260807 06:16:31.380861 6059 catalog_manager.cc:4841] TS 1365b9f4e7584e5e9b7ce7968d2b4355 (127.9.133.131:34439): tablet a0d0cc646a39424f9e9d9a58ba6796db (table test-table [id=1d473cbc7cca41bf870fb2cc995f892f]) successfully deleted
I20260807 06:16:31.381501 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 6336
W20260807 06:16:31.381614 6062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.133.129:39527 (error 108)
I20260807 06:16:31.385435 6693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:31.385020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.129:39527 (local address 127.0.0.1:34874)
0807 06:16:31.385209 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 06:16:31.385300 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.129:39527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 06:16:31.385669 6062 catalog_manager.cc:4584] TS 8f2c44e2cd9a4470977a3eab2abd77a7 (127.9.133.129:39527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0d0cc646a39424f9e9d9a58ba6796db: Network error: Client connection negotiation failed: client connection to 127.9.133.129:39527: connect: Connection refused (error 111)
I20260807 06:16:31.395490 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 6522
I20260807 06:16:31.420631 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 6030
2026-08-07T06:16:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1281 ms)
[----------] 6 tests from EncryptionPolicyTest (12739 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:31 dist-test-slave-h3ft krb5kdc[6744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:31 dist-test-slave-h3ft krb5kdc[6744](info): set up 2 sockets
Aug 07 06:16:31 dist-test-slave-h3ft krb5kdc[6744](info): commencing operation
krb5kdc: starting...
W20260807 06:16:32.705276 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:32 dist-test-slave-h3ft krb5kdc[6744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:32.862430 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:32.997061 6761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:32.997447 6761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:32.997525 6761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:33.001735 6761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:33.001848 6761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:33.001884 6761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:33.001917 6761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:33.001953 6761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:33.007726 6761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:33.009222 6761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:33.010612 6761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:33.021086 6767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:33.021080 6766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:33.021133 6770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:33.021238 6761 server_base.cc:1034] running on GCE node
I20260807 06:16:33.022202 6761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:33.023600 6761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:33.024786 6761 hybrid_clock.cc:648] HybridClock initialized: now 1786083393024745 us; error 52 us; skew 500 ppm
Aug 07 06:16:33 dist-test-slave-h3ft krb5kdc[6744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:33.028240 6761 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:33.029345 6761 webserver.cc:460] Webserver started at http://127.0.0.1:42775/ using document root <none> and password file <none>
I20260807 06:16:33.030050 6761 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:33.030119 6761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:33.030381 6761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:33.033020 6761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "78b072eb202e463e9c762b3d75669a78"
format_stamp: "Formatted at 2026-08-07 06:16:33 on dist-test-slave-h3ft"
server_key: "a7eb3b14e21eb582b182fd08f97e578b"
server_key_iv: "2ed41d914f2ccc554b3fb5365c67780d"
server_key_version: "encryptionkey@0"
I20260807 06:16:33.033720 6761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "78b072eb202e463e9c762b3d75669a78"
format_stamp: "Formatted at 2026-08-07 06:16:33 on dist-test-slave-h3ft"
server_key: "a7eb3b14e21eb582b182fd08f97e578b"
server_key_iv: "2ed41d914f2ccc554b3fb5365c67780d"
server_key_version: "encryptionkey@0"
I20260807 06:16:33.038393 6761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 06:16:33.041592 6781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:33.043222 6761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:33.043396 6761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "78b072eb202e463e9c762b3d75669a78"
format_stamp: "Formatted at 2026-08-07 06:16:33 on dist-test-slave-h3ft"
server_key: "a7eb3b14e21eb582b182fd08f97e578b"
server_key_iv: "2ed41d914f2ccc554b3fb5365c67780d"
server_key_version: "encryptionkey@0"
I20260807 06:16:33.043542 6761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:33.067685 6761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:33.071887 6761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:33.072158 6761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:33.081288 6761 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40417
I20260807 06:16:33.081310 6856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40417 every 8 connection(s)
I20260807 06:16:33.082558 6761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:33.086279 6857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:33.092229 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 6761
I20260807 06:16:33.092422 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:33.092832 9750 external_mini_cluster.cc:1428] Setting key 8dc1113ec8349fa89ba8d722d3547da1
I20260807 06:16:33.094164 6857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78: Bootstrap starting.
I20260807 06:16:33.097333 6857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:33.098332 6857 log.cc:824] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:33.101028 6857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78: No bootstrap required, opened a new log
Aug 07 06:16:33 dist-test-slave-h3ft krb5kdc[6744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:16:33.106856 6857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b072eb202e463e9c762b3d75669a78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260807 06:16:33.107280 6857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:33.107388 6857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b072eb202e463e9c762b3d75669a78, State: Initialized, Role: FOLLOWER
I20260807 06:16:33.108073 6857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b072eb202e463e9c762b3d75669a78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260807 06:16:33.108302 6857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:33.108400 6857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:33.108551 6857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:33.110013 6857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b072eb202e463e9c762b3d75669a78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260807 06:16:33.110532 6857 leader_election.cc:302] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78b072eb202e463e9c762b3d75669a78; no voters:
I20260807 06:16:33.111140 6857 leader_election.cc:288] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:33.112023 6863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:33.112213 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:33.094801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49517 (local address 127.0.0.1:40417)
0807 06:16:33.095395 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:33.095415 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:33.095459 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:16:33.096328 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:33.096353 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:33.096778 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:33.097179 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:33.098190 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:33.099656 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:33.100676 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:33.101291 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:33.101476 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:33.105373 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:33.105380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:33.105402 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:33.105440 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:33.107828 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:33.108594 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:33.108600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:33.108607 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:33.108796 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:33.109235 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:33.109238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:33.109239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:33.109753 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:33.109969 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:33.110494 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 06:16:33.110843 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":179,"threads_started":1}
I20260807 06:16:33.112319 6863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [term 1 LEADER]: Becoming Leader. State: Replica: 78b072eb202e463e9c762b3d75669a78, State: Running, Role: LEADER
I20260807 06:16:33.112769 6863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b072eb202e463e9c762b3d75669a78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260807 06:16:33.112951 6857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:33.114931 6865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78b072eb202e463e9c762b3d75669a78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b072eb202e463e9c762b3d75669a78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } } }
I20260807 06:16:33.115126 6865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:33.115628 6864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78b072eb202e463e9c762b3d75669a78. Latest consensus state: current_term: 1 leader_uuid: "78b072eb202e463e9c762b3d75669a78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b072eb202e463e9c762b3d75669a78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } } }
I20260807 06:16:33.115849 6864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:33.117467 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 06:16:33.120648 6881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:33.120750 6881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:33.120834 6872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:33.122037 6872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:33.130415 6872 catalog_manager.cc:1347] Generated new cluster ID: 37054c60b39f4287bed7f54203df15a1
I20260807 06:16:33.130618 6872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:33.132045 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:33.120516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52321 (local address 127.0.0.1:40417)
0807 06:16:33.120677 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:33.120685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:33.121167 (+ 482us) server_negotiation.cc:407] Connection header received
0807 06:16:33.121321 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:33.121325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:33.121394 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:33.121493 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:33.122552 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:33.123473 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:33.124719 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:33.124995 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:33.125124 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:33.126964 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:33.126968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:33.126972 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:33.127057 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:33.129517 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:33.130247 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:33.130251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:33.130254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:33.130372 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:33.130889 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:33.130893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:33.130894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:33.131205 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:33.131405 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:33.131722 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 06:16:33.131848 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:16:33.143726 6872 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:33.145556 6872 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:33.155423 6872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78b072eb202e463e9c762b3d75669a78: Generated new TSK 0
I20260807 06:16:33.156388 6872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:33.165815 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 6761
W20260807 06:16:33.181375 6742 connection.cc:535] client connection to 127.0.0.1:40417 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40417): Connection reset by peer (error 104)
2026-08-07T06:16:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1745 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:33 dist-test-slave-h3ft krb5kdc[6894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:33 dist-test-slave-h3ft krb5kdc[6894](info): set up 2 sockets
Aug 07 06:16:33 dist-test-slave-h3ft krb5kdc[6894](info): commencing operation
krb5kdc: starting...
W20260807 06:16:34.493242 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.007s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:34 dist-test-slave-h3ft krb5kdc[6894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:34.642529 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:34.778167 6908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:34.778576 6908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:34.778659 6908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:34.782933 6908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:34.783087 6908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:34.783129 6908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:34.783185 6908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:34.783242 6908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:34.789266 6908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:34.790694 6908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:34.792032 6908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:34.803224 6920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:34.803224 6915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:34.803418 6908 server_base.cc:1034] running on GCE node
W20260807 06:16:34.803640 6917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:34.804481 6908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:34.805958 6908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:34.807111 6908 hybrid_clock.cc:648] HybridClock initialized: now 1786083394807081 us; error 37 us; skew 500 ppm
Aug 07 06:16:34 dist-test-slave-h3ft krb5kdc[6894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:34.816505 6908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:34.817835 6908 webserver.cc:460] Webserver started at http://127.0.0.1:33043/ using document root <none> and password file <none>
I20260807 06:16:34.818624 6908 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:34.818737 6908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:34.819118 6908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:34.821828 6908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4"
format_stamp: "Formatted at 2026-08-07 06:16:34 on dist-test-slave-h3ft"
server_key: "e353cf526f625bb9e7a34376648504c0"
server_key_iv: "1c717dd277c8dfa56bdb830bb5fb4196"
server_key_version: "encryptionkey@0"
I20260807 06:16:34.822547 6908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4"
format_stamp: "Formatted at 2026-08-07 06:16:34 on dist-test-slave-h3ft"
server_key: "e353cf526f625bb9e7a34376648504c0"
server_key_iv: "1c717dd277c8dfa56bdb830bb5fb4196"
server_key_version: "encryptionkey@0"
I20260807 06:16:34.827373 6908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260807 06:16:34.830503 6928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:34.832217 6908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:34.832404 6908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4"
format_stamp: "Formatted at 2026-08-07 06:16:34 on dist-test-slave-h3ft"
server_key: "e353cf526f625bb9e7a34376648504c0"
server_key_iv: "1c717dd277c8dfa56bdb830bb5fb4196"
server_key_version: "encryptionkey@0"
I20260807 06:16:34.832546 6908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:34.859436 6908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:34.864135 6908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:34.864504 6908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:34.874354 6908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38295
I20260807 06:16:34.874388 6996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38295 every 8 connection(s)
I20260807 06:16:34.875769 6908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:34.879598 6999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:34.882267 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 6908
I20260807 06:16:34.882448 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:34.882822 9750 external_mini_cluster.cc:1428] Setting key c979e57845487193cd89695c4eaf2eea
I20260807 06:16:34.888140 6999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4: Bootstrap starting.
I20260807 06:16:34.890965 6999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:34.891981 6999 log.cc:824] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:34.893622 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:34.884685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49589 (local address 127.0.0.1:38295)
0807 06:16:34.885274 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:34.885292 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:34.885337 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:16:34.886113 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:34.886136 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:34.886550 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:34.886962 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:34.887480 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:34.888620 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:34.889597 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:34.890116 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:34.890308 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:34.891445 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:34.891450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:34.891465 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:34.891936 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:34.892001 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:34.892025 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:34.892242 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:34.892494 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 06:16:34.892692 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":177,"threads_started":1}
I20260807 06:16:34.894645 6999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4: No bootstrap required, opened a new log
I20260807 06:16:34.897046 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:34.899477 6999 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260807 06:16:34.899869 6999 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:34.899963 6999 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ec3c51ac974b5fb01dd73f97ea82f4, State: Initialized, Role: FOLLOWER
I20260807 06:16:34.900548 6999 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260807 06:16:34.900774 6999 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:34.900924 6999 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:34.901055 6999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:34.902491 6999 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260807 06:16:34.902994 6999 leader_election.cc:302] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6ec3c51ac974b5fb01dd73f97ea82f4; no voters:
I20260807 06:16:34.903467 6999 leader_election.cc:288] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:34.903622 7016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:34.903872 7016 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [term 1 LEADER]: Becoming Leader. State: Replica: b6ec3c51ac974b5fb01dd73f97ea82f4, State: Running, Role: LEADER
I20260807 06:16:34.904322 7016 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260807 06:16:34.904938 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:34.899527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53519 (local address 127.0.0.1:38295)
0807 06:16:34.899675 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:34.899679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:34.900081 (+ 402us) server_negotiation.cc:407] Connection header received
0807 06:16:34.900127 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:34.900131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:34.900183 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:34.900273 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:34.900647 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:34.901569 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:34.902521 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:34.902877 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:34.902994 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:34.904225 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:34.904229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:34.904232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:34.904248 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:34.904279 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:34.904293 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:34.904384 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:34.904758 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 06:16:34.904798 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 06:16:34.905272 6999 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:34.906586 7017 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } } }
I20260807 06:16:34.906744 7017 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:34.907248 7031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:34.907920 7018 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6ec3c51ac974b5fb01dd73f97ea82f4. Latest consensus state: current_term: 1 leader_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ec3c51ac974b5fb01dd73f97ea82f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } } }
I20260807 06:16:34.908064 7018 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:34.911911 7031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:34.919728 7031 catalog_manager.cc:1347] Generated new cluster ID: 7de8c5245eda4503806991e4641c5178
I20260807 06:16:34.919828 7031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:34.936236 7031 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:34.938126 7031 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:34.945199 7031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6ec3c51ac974b5fb01dd73f97ea82f4: Generated new TSK 0
I20260807 06:16:34.946400 7031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:34.963532 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 6908
W20260807 06:16:34.977804 6892 connection.cc:535] client connection to 127.0.0.1:38295 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38295): Connection reset by peer (error 104)
2026-08-07T06:16:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1796 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:35 dist-test-slave-h3ft krb5kdc[7047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:35 dist-test-slave-h3ft krb5kdc[7047](info): set up 2 sockets
Aug 07 06:16:35 dist-test-slave-h3ft krb5kdc[7047](info): commencing operation
krb5kdc: starting...
W20260807 06:16:36.237185 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:36 dist-test-slave-h3ft krb5kdc[7047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:36.390916 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:36.522861 7065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:36.523241 7065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:36.523313 7065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:36.527343 7065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:36.527410 7065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:36.527429 7065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:36.527446 7065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:36.527462 7065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:36.533095 7065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:36.534471 7065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:36.535826 7065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:36.546197 7074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:36.546373 7065 server_base.cc:1034] running on GCE node
W20260807 06:16:36.546243 7072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:36.546674 7077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:36.547474 7065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:36.548815 7065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:36.550021 7065 hybrid_clock.cc:648] HybridClock initialized: now 1786083396549989 us; error 53 us; skew 500 ppm
Aug 07 06:16:36 dist-test-slave-h3ft krb5kdc[7047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:36.553462 7065 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:36.554596 7065 webserver.cc:460] Webserver started at http://127.0.0.1:33677/ using document root <none> and password file <none>
I20260807 06:16:36.555404 7065 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:36.555502 7065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:36.555778 7065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:36.558322 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba"
format_stamp: "Formatted at 2026-08-07 06:16:36 on dist-test-slave-h3ft"
server_key: "06cb1e2aa5d8f3324f497384bc96703c"
server_key_iv: "a96fac618819a2fd01c290517bf40be9"
server_key_version: "encryptionkey@0"
I20260807 06:16:36.559091 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba"
format_stamp: "Formatted at 2026-08-07 06:16:36 on dist-test-slave-h3ft"
server_key: "06cb1e2aa5d8f3324f497384bc96703c"
server_key_iv: "a96fac618819a2fd01c290517bf40be9"
server_key_version: "encryptionkey@0"
I20260807 06:16:36.563685 7065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 06:16:36.566555 7085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:36.568109 7065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:36.568271 7065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba"
format_stamp: "Formatted at 2026-08-07 06:16:36 on dist-test-slave-h3ft"
server_key: "06cb1e2aa5d8f3324f497384bc96703c"
server_key_iv: "a96fac618819a2fd01c290517bf40be9"
server_key_version: "encryptionkey@0"
I20260807 06:16:36.568401 7065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:36.573964 7065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:36.577597 7065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:36.577834 7065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:36.586681 7065 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33421
I20260807 06:16:36.586711 7154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33421 every 8 connection(s)
I20260807 06:16:36.587859 7065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:36.590161 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7065
I20260807 06:16:36.590337 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:36.590698 9750 external_mini_cluster.cc:1428] Setting key 2ce134008ff2d918656359ae96bc5a16
I20260807 06:16:36.591660 7156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:36.596822 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:36.592421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51369 (local address 127.0.0.1:33421)
0807 06:16:36.592906 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:36.592926 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:36.592980 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:16:36.593737 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:36.593756 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:36.594081 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:36.594396 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:36.595400 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:36.595405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:36.595417 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:36.595428 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:36.595504 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:36.595527 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:36.595647 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:36.595783 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 06:16:36.595977 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":171,"threads_started":1}
I20260807 06:16:36.599171 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba: Bootstrap starting.
I20260807 06:16:36.600023 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:36.602447 7156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:36.603591 7156 log.cc:824] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:36.605357 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:36.602720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44483 (local address 127.0.0.1:33421)
0807 06:16:36.602831 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:36.602835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:36.603398 (+ 563us) server_negotiation.cc:407] Connection header received
0807 06:16:36.603583 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:36.603589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:36.603676 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:36.603800 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:36.604845 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:36.604849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:36.604851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:36.604887 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:36.604919 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:36.604933 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:36.605034 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:36.605190 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 06:16:36.605231 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 06:16:36.606357 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba: No bootstrap required, opened a new log
I20260807 06:16:36.610745 7156 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260807 06:16:36.611069 7156 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:36.611133 7156 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8df9bff5ac64ed1a1a49bec84eaf4ba, State: Initialized, Role: FOLLOWER
I20260807 06:16:36.611630 7156 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260807 06:16:36.611768 7156 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:36.611819 7156 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:36.611909 7156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:36.612962 7156 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260807 06:16:36.613312 7156 leader_election.cc:302] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8df9bff5ac64ed1a1a49bec84eaf4ba; no voters:
I20260807 06:16:36.613617 7156 leader_election.cc:288] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:36.613818 7166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:36.614224 7166 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [term 1 LEADER]: Becoming Leader. State: Replica: d8df9bff5ac64ed1a1a49bec84eaf4ba, State: Running, Role: LEADER
I20260807 06:16:36.614603 7166 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260807 06:16:36.615211 7156 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:36.616708 7168 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8df9bff5ac64ed1a1a49bec84eaf4ba. Latest consensus state: current_term: 1 leader_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } } }
I20260807 06:16:36.616868 7168 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:36.617302 7167 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8df9bff5ac64ed1a1a49bec84eaf4ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } } }
I20260807 06:16:36.617431 7167 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba [sys.catalog]: This master's current role is: LEADER
W20260807 06:16:36.620757 7187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:36.620833 7187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:36.620926 7181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:36.621703 7181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:36.628214 7181 catalog_manager.cc:1347] Generated new cluster ID: 34aeff3306954691a1a35b0db81e285d
I20260807 06:16:36.628293 7181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:36.665495 7181 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:36.666018 7181 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:36.672019 7181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8df9bff5ac64ed1a1a49bec84eaf4ba: Generated new TSK 0
I20260807 06:16:36.673141 7181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:36.688145 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7065
2026-08-07T06:16:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1724 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:36 dist-test-slave-h3ft krb5kdc[7197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:36 dist-test-slave-h3ft krb5kdc[7197](info): set up 2 sockets
Aug 07 06:16:36 dist-test-slave-h3ft krb5kdc[7197](info): commencing operation
krb5kdc: starting...
W20260807 06:16:38.204564 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.486s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:38 dist-test-slave-h3ft krb5kdc[7197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:38.363485 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:38.502911 7213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:38.503393 7213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:38.503507 7213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:38.507912 7213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:38.508013 7213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:38.508054 7213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:38.508114 7213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:38.508152 7213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:38.513963 7213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:38.515517 7213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:38.516880 7213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:38.527305 7223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:38.527295 7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:38.527295 7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:38.527870 7213 server_base.cc:1034] running on GCE node
I20260807 06:16:38.528574 7213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:38.530031 7213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:38.531224 7213 hybrid_clock.cc:648] HybridClock initialized: now 1786083398531201 us; error 40 us; skew 500 ppm
Aug 07 06:16:38 dist-test-slave-h3ft krb5kdc[7197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:38.535221 7213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:38.536522 7213 webserver.cc:460] Webserver started at http://127.0.0.1:34945/ using document root <none> and password file <none>
I20260807 06:16:38.537299 7213 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:38.537428 7213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:38.537750 7213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:38.540642 7213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "a3d5041e508e4037b1377c4846a674f3"
format_stamp: "Formatted at 2026-08-07 06:16:38 on dist-test-slave-h3ft"
server_key: "f1c52ae9b71158c434f5a77520c2c7b9"
server_key_iv: "b543ccf39f6f45ea089a4055b914b1ae"
server_key_version: "encryptionkey@0"
I20260807 06:16:38.541384 7213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "a3d5041e508e4037b1377c4846a674f3"
format_stamp: "Formatted at 2026-08-07 06:16:38 on dist-test-slave-h3ft"
server_key: "f1c52ae9b71158c434f5a77520c2c7b9"
server_key_iv: "b543ccf39f6f45ea089a4055b914b1ae"
server_key_version: "encryptionkey@0"
I20260807 06:16:38.546397 7213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:16:38.549690 7236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:38.551440 7213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:16:38.551609 7213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "a3d5041e508e4037b1377c4846a674f3"
format_stamp: "Formatted at 2026-08-07 06:16:38 on dist-test-slave-h3ft"
server_key: "f1c52ae9b71158c434f5a77520c2c7b9"
server_key_iv: "b543ccf39f6f45ea089a4055b914b1ae"
server_key_version: "encryptionkey@0"
I20260807 06:16:38.551753 7213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:38.582361 7213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:38.586709 7213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:38.586987 7213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:38.596477 7213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44349
I20260807 06:16:38.596501 7301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44349 every 8 connection(s)
I20260807 06:16:38.597703 7213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:38.601356 7303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:38.603585 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7213
I20260807 06:16:38.603770 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:38.604146 9750 external_mini_cluster.cc:1428] Setting key dbef00c39d3b72ee1edf8d5f0ae8ed93
I20260807 06:16:38.609699 7303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3: Bootstrap starting.
I20260807 06:16:38.612818 7303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:38.613767 7303 log.cc:824] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3: Log is configured to *not* fsync() on all Append() calls
Aug 07 06:16:38 dist-test-slave-h3ft krb5kdc[7197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:16:38.616847 7303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3: No bootstrap required, opened a new log
I20260807 06:16:38.621486 7303 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d5041e508e4037b1377c4846a674f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260807 06:16:38.621819 7303 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:38.621932 7303 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d5041e508e4037b1377c4846a674f3, State: Initialized, Role: FOLLOWER
I20260807 06:16:38.622354 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:38.606091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49145 (local address 127.0.0.1:44349)
0807 06:16:38.606670 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:38.606700 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:38.606759 (+ 59us) server_negotiation.cc:407] Connection header received
0807 06:16:38.607672 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:38.607697 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:38.608156 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:38.608590 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:38.609632 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:38.610816 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:38.611858 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:38.612409 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:38.612610 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:38.615908 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:38.615916 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:38.615938 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:38.615975 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:38.618553 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:38.619243 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:38.619250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:38.619258 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:38.619404 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:38.619727 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:38.619730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:38.619732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:38.620229 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:38.620481 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:38.620802 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 06:16:38.621210 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":172,"threads_started":1}
I20260807 06:16:38.622659 7303 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d5041e508e4037b1377c4846a674f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260807 06:16:38.622838 7303 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:38.622932 7303 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:38.623080 7303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:38.624467 7303 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d5041e508e4037b1377c4846a674f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260807 06:16:38.624893 7303 leader_election.cc:302] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3d5041e508e4037b1377c4846a674f3; no voters:
I20260807 06:16:38.625303 7303 leader_election.cc:288] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:38.625435 7309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:38.625739 7309 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [term 1 LEADER]: Becoming Leader. State: Replica: a3d5041e508e4037b1377c4846a674f3, State: Running, Role: LEADER
I20260807 06:16:38.626119 7309 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d5041e508e4037b1377c4846a674f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } }
I20260807 06:16:38.626387 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:38.626966 7303 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:38.628392 7311 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3d5041e508e4037b1377c4846a674f3. Latest consensus state: current_term: 1 leader_uuid: "a3d5041e508e4037b1377c4846a674f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d5041e508e4037b1377c4846a674f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } } }
I20260807 06:16:38.628588 7311 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:38.629541 7310 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3d5041e508e4037b1377c4846a674f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d5041e508e4037b1377c4846a674f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44349 } } }
I20260807 06:16:38.629645 7310 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3 [sys.catalog]: This master's current role is: LEADER
W20260807 06:16:38.634109 7332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:38.634200 7332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:38.634296 7322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:38.635299 7322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:38.642037 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:38.630186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59161 (local address 127.0.0.1:44349)
0807 06:16:38.630320 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:38.630326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:38.631246 (+ 920us) server_negotiation.cc:407] Connection header received
0807 06:16:38.631486 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:38.631490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:38.631556 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:38.631628 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:38.632573 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:38.633210 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:38.634150 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:38.634542 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:38.634675 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:38.636381 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:38.636385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:38.636388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:38.636424 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:38.639860 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:38.640365 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:38.640370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:38.640375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:38.640476 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:38.640840 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:38.640844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:38.640854 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:38.641141 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:38.641305 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:38.641645 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 06:16:38.641810 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 06:16:38.642881 7322 catalog_manager.cc:1347] Generated new cluster ID: 06be22e7dac449b8bf6e4f2dc1f3e4d1
I20260807 06:16:38.642962 7322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:38.657558 7322 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:38.658782 7322 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:38.665518 7322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3d5041e508e4037b1377c4846a674f3: Generated new TSK 0
I20260807 06:16:38.666936 7322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:38.675094 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7213
2026-08-07T06:16:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1987 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:38 dist-test-slave-h3ft krb5kdc[7341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:38 dist-test-slave-h3ft krb5kdc[7341](info): set up 2 sockets
Aug 07 06:16:38 dist-test-slave-h3ft krb5kdc[7341](info): commencing operation
krb5kdc: starting...
W20260807 06:16:39.967538 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:40 dist-test-slave-h3ft krb5kdc[7341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:40.134928 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:40.270962 7357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:40.271284 7357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:40.271327 7357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:40.275454 7357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:40.275517 7357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:40.275537 7357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:40.275554 7357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:40.275590 7357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:40.281075 7357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:40.282369 7357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:40.283908 7357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:40.294328 7366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:40.294413 7357 server_base.cc:1034] running on GCE node
W20260807 06:16:40.294382 7364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:40.294339 7363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:40.295522 7357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:40.296823 7357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:40.298027 7357 hybrid_clock.cc:648] HybridClock initialized: now 1786083400297993 us; error 41 us; skew 500 ppm
Aug 07 06:16:40 dist-test-slave-h3ft krb5kdc[7341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:40.307765 7357 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:40.308984 7357 webserver.cc:460] Webserver started at http://127.0.0.1:44505/ using document root <none> and password file <none>
I20260807 06:16:40.309701 7357 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:40.309803 7357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:40.310083 7357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:40.312652 7357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "8272574e445345d69d6c6c38464ad6bc"
format_stamp: "Formatted at 2026-08-07 06:16:40 on dist-test-slave-h3ft"
server_key: "df0eb50d09bc838287254bed5a154371"
server_key_iv: "56fa02d885e15606f6ee874e4679ff17"
server_key_version: "encryptionkey@0"
I20260807 06:16:40.313308 7357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "8272574e445345d69d6c6c38464ad6bc"
format_stamp: "Formatted at 2026-08-07 06:16:40 on dist-test-slave-h3ft"
server_key: "df0eb50d09bc838287254bed5a154371"
server_key_iv: "56fa02d885e15606f6ee874e4679ff17"
server_key_version: "encryptionkey@0"
I20260807 06:16:40.317728 7357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:16:40.320659 7376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:40.322077 7357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:16:40.322235 7357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "8272574e445345d69d6c6c38464ad6bc"
format_stamp: "Formatted at 2026-08-07 06:16:40 on dist-test-slave-h3ft"
server_key: "df0eb50d09bc838287254bed5a154371"
server_key_iv: "56fa02d885e15606f6ee874e4679ff17"
server_key_version: "encryptionkey@0"
I20260807 06:16:40.322364 7357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:40.348909 7357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:40.353130 7357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:40.353427 7357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:40.362099 7357 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46481
I20260807 06:16:40.362128 7453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46481 every 8 connection(s)
I20260807 06:16:40.363261 7357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:40.365561 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7357
I20260807 06:16:40.365741 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:40.366096 9750 external_mini_cluster.cc:1428] Setting key f5249f272396a9a8ad0f61c7703f695b
I20260807 06:16:40.366925 7455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:40.373685 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc: Bootstrap starting.
I20260807 06:16:40.376333 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:40.367766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38075 (local address 127.0.0.1:46481)
0807 06:16:40.368312 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:40.368329 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:40.368377 (+ 48us) server_negotiation.cc:407] Connection header received
0807 06:16:40.369194 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:40.369215 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:40.369603 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:40.370010 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:40.370481 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:40.371658 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:40.372658 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:40.373165 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:40.373371 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:40.374367 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:40.374372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:40.374386 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:40.374824 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:40.374931 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:40.374958 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:40.375117 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:40.375275 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 06:16:40.375514 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":185,"threads_started":1}
I20260807 06:16:40.376518 7455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:40.377359 7455 log.cc:824] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:40.379662 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:40.379949 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc: No bootstrap required, opened a new log
I20260807 06:16:40.384740 7455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8272574e445345d69d6c6c38464ad6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46481 } }
I20260807 06:16:40.385100 7455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:40.385190 7455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8272574e445345d69d6c6c38464ad6bc, State: Initialized, Role: FOLLOWER
I20260807 06:16:40.385803 7455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8272574e445345d69d6c6c38464ad6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46481 } }
I20260807 06:16:40.385989 7455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:40.386071 7455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:40.386184 7455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:40.386662 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:40.382002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41029 (local address 127.0.0.1:46481)
0807 06:16:40.382113 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:40.382116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:40.382371 (+ 255us) server_negotiation.cc:407] Connection header received
0807 06:16:40.382487 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:40.382490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:40.382528 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:40.382588 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:40.382909 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:40.383469 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:40.384587 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:40.384887 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:40.385000 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:40.386081 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:40.386086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:40.386089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:40.386107 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:40.386167 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:40.386185 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:40.386284 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:40.386431 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 06:16:40.386491 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 06:16:40.387516 7455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8272574e445345d69d6c6c38464ad6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46481 } }
I20260807 06:16:40.387940 7455 leader_election.cc:302] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8272574e445345d69d6c6c38464ad6bc; no voters:
I20260807 06:16:40.388303 7455 leader_election.cc:288] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:40.388489 7468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:40.388772 7468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [term 1 LEADER]: Becoming Leader. State: Replica: 8272574e445345d69d6c6c38464ad6bc, State: Running, Role: LEADER
I20260807 06:16:40.389292 7468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8272574e445345d69d6c6c38464ad6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46481 } }
I20260807 06:16:40.389709 7455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:40.391399 7469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8272574e445345d69d6c6c38464ad6bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8272574e445345d69d6c6c38464ad6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46481 } } }
I20260807 06:16:40.391422 7470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8272574e445345d69d6c6c38464ad6bc. Latest consensus state: current_term: 1 leader_uuid: "8272574e445345d69d6c6c38464ad6bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8272574e445345d69d6c6c38464ad6bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46481 } } }
I20260807 06:16:40.391566 7470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:40.391563 7469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:40.392023 7482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:40.396060 7482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:40.402932 7482 catalog_manager.cc:1347] Generated new cluster ID: 48b72d2b190c44a3ab97ec759f281d6b
I20260807 06:16:40.403051 7482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:40.429258 7482 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:40.430496 7482 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:40.436308 7482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8272574e445345d69d6c6c38464ad6bc: Generated new TSK 0
I20260807 06:16:40.437206 7482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:40.449957 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7357
2026-08-07T06:16:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1772 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:40 dist-test-slave-h3ft krb5kdc[7494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:40 dist-test-slave-h3ft krb5kdc[7494](info): set up 2 sockets
Aug 07 06:16:40 dist-test-slave-h3ft krb5kdc[7494](info): commencing operation
krb5kdc: starting...
W20260807 06:16:41.716723 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:41 dist-test-slave-h3ft krb5kdc[7494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:41.870962 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39263
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:42.002494 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:42.002887 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:42.002972 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:42.007118 7509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:42.007229 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:42.007266 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:42.007305 7509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:42.007336 7509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:42.012988 7509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:42.014369 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:42.015712 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:42.026317 7515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:42.026293 7516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:42.026293 7518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:42.026360 7509 server_base.cc:1034] running on GCE node
I20260807 06:16:42.027487 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:42.028952 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:42.030179 7509 hybrid_clock.cc:648] HybridClock initialized: now 1786083402030167 us; error 35 us; skew 500 ppm
Aug 07 06:16:42 dist-test-slave-h3ft krb5kdc[7494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083402, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:42.033793 7509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:42.035055 7509 webserver.cc:460] Webserver started at http://127.0.0.1:44833/ using document root <none> and password file <none>
I20260807 06:16:42.035759 7509 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:42.035825 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:42.036039 7509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:42.038539 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "759a8b9329eb4be684ab03f1b8cdb71f"
format_stamp: "Formatted at 2026-08-07 06:16:42 on dist-test-slave-h3ft"
server_key: "c3f3c1a10b19de673f33006084ea921b"
server_key_iv: "2b583bd331ac97810724c833291370fa"
server_key_version: "encryptionkey@0"
I20260807 06:16:42.039235 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "759a8b9329eb4be684ab03f1b8cdb71f"
format_stamp: "Formatted at 2026-08-07 06:16:42 on dist-test-slave-h3ft"
server_key: "c3f3c1a10b19de673f33006084ea921b"
server_key_iv: "2b583bd331ac97810724c833291370fa"
server_key_version: "encryptionkey@0"
I20260807 06:16:42.043699 7509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:42.046454 7526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:42.047942 7509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:42.048058 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "759a8b9329eb4be684ab03f1b8cdb71f"
format_stamp: "Formatted at 2026-08-07 06:16:42 on dist-test-slave-h3ft"
server_key: "c3f3c1a10b19de673f33006084ea921b"
server_key_iv: "2b583bd331ac97810724c833291370fa"
server_key_version: "encryptionkey@0"
I20260807 06:16:42.048156 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:42.053627 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:42.057417 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:42.057598 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:42.066802 7509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39263
I20260807 06:16:42.066820 7600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39263 every 8 connection(s)
I20260807 06:16:42.068161 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:42.069417 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7509
I20260807 06:16:42.069604 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:42.069955 9750 external_mini_cluster.cc:1428] Setting key e9d9eb8b2133f44d15192a4aaec0b831
I20260807 06:16:42.072723 7601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:42.076684 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:42.071833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57395 (local address 127.0.0.1:39263)
0807 06:16:42.072405 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:42.072423 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:42.072467 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:16:42.073321 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:42.073345 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:42.073712 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:42.074142 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:42.075195 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:42.075201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:42.075215 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:42.075227 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:42.075295 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:42.075318 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:42.075498 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:42.075579 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 06:16:42.075774 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":189,"threads_started":1}
I20260807 06:16:42.080734 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:42.081637 7601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f: Bootstrap starting.
I20260807 06:16:42.085685 7601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:42.086190 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:42.083962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34219 (local address 127.0.0.1:39263)
0807 06:16:42.084117 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:42.084123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:42.084416 (+ 293us) server_negotiation.cc:407] Connection header received
0807 06:16:42.084588 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:42.084590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:42.084631 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:42.084704 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:42.085621 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:42.085625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:42.085628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:42.085638 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:42.085676 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:42.085687 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:42.085758 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:42.085964 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 06:16:42.086021 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:16:42.087035 7601 log.cc:824] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:42.089988 7601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f: No bootstrap required, opened a new log
I20260807 06:16:42.094558 7601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260807 06:16:42.094870 7601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:42.094911 7601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 759a8b9329eb4be684ab03f1b8cdb71f, State: Initialized, Role: FOLLOWER
I20260807 06:16:42.095530 7601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260807 06:16:42.095681 7601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:42.095736 7601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:42.095824 7601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:42.096980 7601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260807 06:16:42.097368 7601 leader_election.cc:302] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 759a8b9329eb4be684ab03f1b8cdb71f; no voters:
I20260807 06:16:42.097739 7601 leader_election.cc:288] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:42.097875 7615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:42.098176 7615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [term 1 LEADER]: Becoming Leader. State: Replica: 759a8b9329eb4be684ab03f1b8cdb71f, State: Running, Role: LEADER
I20260807 06:16:42.098649 7615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } }
I20260807 06:16:42.099561 7601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:42.101075 7617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 759a8b9329eb4be684ab03f1b8cdb71f. Latest consensus state: current_term: 1 leader_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } } }
I20260807 06:16:42.101197 7616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759a8b9329eb4be684ab03f1b8cdb71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39263 } } }
I20260807 06:16:42.101315 7616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:42.101315 7617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:42.101874 7626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:42.105636 7626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:42.112620 7626 catalog_manager.cc:1347] Generated new cluster ID: b76f2414a0344467ae4526a5aaa85cc0
I20260807 06:16:42.112718 7626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:42.126746 7626 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:42.127233 7626 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:42.140879 7626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 759a8b9329eb4be684ab03f1b8cdb71f: Generated new TSK 0
I20260807 06:16:42.142160 7626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:42.147190 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7509
2026-08-07T06:16:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1696 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:42 dist-test-slave-h3ft krb5kdc[7645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:42 dist-test-slave-h3ft krb5kdc[7645](info): set up 2 sockets
Aug 07 06:16:42 dist-test-slave-h3ft krb5kdc[7645](info): commencing operation
krb5kdc: starting...
W20260807 06:16:43.411496 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:43 dist-test-slave-h3ft krb5kdc[7645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:43.559542 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:43.697645 7664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:43.698185 7664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:43.698282 7664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:43.702585 7664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:43.702656 7664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:43.702679 7664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:43.702697 7664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:43.702713 7664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:43.708526 7664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:43.709829 7664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:43.711153 7664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:43.721688 7673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:43.721681 7670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:43.721647 7671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:43.722086 7664 server_base.cc:1034] running on GCE node
I20260807 06:16:43.722919 7664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:43.724462 7664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:43.725706 7664 hybrid_clock.cc:648] HybridClock initialized: now 1786083403725676 us; error 44 us; skew 500 ppm
Aug 07 06:16:43 dist-test-slave-h3ft krb5kdc[7645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:43.729604 7664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:43.730854 7664 webserver.cc:460] Webserver started at http://127.0.0.1:38939/ using document root <none> and password file <none>
I20260807 06:16:43.731694 7664 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:43.731806 7664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:43.732123 7664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:43.734824 7664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "c1f5abf5b5604329aa54f3121b7f2022"
format_stamp: "Formatted at 2026-08-07 06:16:43 on dist-test-slave-h3ft"
server_key: "b3e7fb2d2e9c43d8c7863cde799e2b22"
server_key_iv: "075a50dd49eb84e4476a0d692be6d248"
server_key_version: "encryptionkey@0"
I20260807 06:16:43.735621 7664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "c1f5abf5b5604329aa54f3121b7f2022"
format_stamp: "Formatted at 2026-08-07 06:16:43 on dist-test-slave-h3ft"
server_key: "b3e7fb2d2e9c43d8c7863cde799e2b22"
server_key_iv: "075a50dd49eb84e4476a0d692be6d248"
server_key_version: "encryptionkey@0"
I20260807 06:16:43.740453 7664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:43.743757 7682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:43.745455 7664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:16:43.745641 7664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "c1f5abf5b5604329aa54f3121b7f2022"
format_stamp: "Formatted at 2026-08-07 06:16:43 on dist-test-slave-h3ft"
server_key: "b3e7fb2d2e9c43d8c7863cde799e2b22"
server_key_iv: "075a50dd49eb84e4476a0d692be6d248"
server_key_version: "encryptionkey@0"
I20260807 06:16:43.745790 7664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:43.763752 7664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:43.767696 7664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:43.767952 7664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:43.777109 7664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43935
I20260807 06:16:43.777140 7758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43935 every 8 connection(s)
I20260807 06:16:43.778257 7664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:43.778501 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7664
I20260807 06:16:43.778666 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:43.779122 9750 external_mini_cluster.cc:1428] Setting key 99cdd10704b669f2edac16f453b40108
I20260807 06:16:43.781925 7760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:43.788966 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022: Bootstrap starting.
Aug 07 06:16:43 dist-test-slave-h3ft krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:16:43.792089 7760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:43.793006 7760 log.cc:824] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:43.795941 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022: No bootstrap required, opened a new log
I20260807 06:16:43.796380 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:43.780942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:34183 (local address 127.0.0.1:43935)
0807 06:16:43.781621 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:43.781650 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:43.781696 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:16:43.782570 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:43.782594 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:43.783088 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:43.783508 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:43.784489 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:43.785590 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:43.786577 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:43.787176 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:43.787355 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:43.790365 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:43.790370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:43.790386 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:43.790417 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:43.792479 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:43.793160 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:43.793166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:43.793172 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:43.793304 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:43.793782 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:43.793785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:43.793786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:43.794363 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:43.794614 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:43.794955 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 06:16:43.795249 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":196,"threads_started":1}
I20260807 06:16:43.799938 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:43.800450 7760 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f5abf5b5604329aa54f3121b7f2022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260807 06:16:43.800809 7760 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:43.801016 7760 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f5abf5b5604329aa54f3121b7f2022, State: Initialized, Role: FOLLOWER
I20260807 06:16:43.801607 7760 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f5abf5b5604329aa54f3121b7f2022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260807 06:16:43.801819 7760 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:43.801944 7760 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:43.802075 7760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:43.803421 7760 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f5abf5b5604329aa54f3121b7f2022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260807 06:16:43.803838 7760 leader_election.cc:302] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f5abf5b5604329aa54f3121b7f2022; no voters:
I20260807 06:16:43.804263 7760 leader_election.cc:288] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:43.804354 7772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:43.804600 7772 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [term 1 LEADER]: Becoming Leader. State: Replica: c1f5abf5b5604329aa54f3121b7f2022, State: Running, Role: LEADER
I20260807 06:16:43.805001 7772 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f5abf5b5604329aa54f3121b7f2022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260807 06:16:43.805913 7760 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:43.807062 7774 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f5abf5b5604329aa54f3121b7f2022. Latest consensus state: current_term: 1 leader_uuid: "c1f5abf5b5604329aa54f3121b7f2022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f5abf5b5604329aa54f3121b7f2022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } } }
I20260807 06:16:43.807204 7774 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:43.807693 7773 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1f5abf5b5604329aa54f3121b7f2022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f5abf5b5604329aa54f3121b7f2022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } } }
I20260807 06:16:43.807886 7773 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:43.808688 7779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:43.812247 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:43.802720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:60613 (local address 127.0.0.1:43935)
0807 06:16:43.802871 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:43.802877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:43.803206 (+ 329us) server_negotiation.cc:407] Connection header received
0807 06:16:43.803375 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:43.803378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:43.803437 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:43.803521 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:43.804334 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:43.805130 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:43.806117 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:43.806432 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:43.806994 (+ 562us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:43.808456 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:43.808463 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:43.808467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:43.808501 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:43.810327 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:43.810853 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:43.810856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:43.810863 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:43.810949 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:43.811332 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:43.811336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:43.811337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:43.811527 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:43.811691 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:43.811982 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 06:16:43.812081 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:16:43.812657 7779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:43.819530 7779 catalog_manager.cc:1347] Generated new cluster ID: 7db86a6a76364a4f9178014c34df14aa
I20260807 06:16:43.819625 7779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:43.851508 7779 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:43.852962 7779 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:43.863538 7779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1f5abf5b5604329aa54f3121b7f2022: Generated new TSK 0
I20260807 06:16:43.864666 7779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:43.881915 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7664
W20260807 06:16:43.901317 7640 connection.cc:535] client connection to 127.0.0.1:43935 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43935): Connection reset by peer (error 104)
2026-08-07T06:16:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1740 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:43 dist-test-slave-h3ft krb5kdc[7802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:43 dist-test-slave-h3ft krb5kdc[7802](info): set up 2 sockets
Aug 07 06:16:43 dist-test-slave-h3ft krb5kdc[7802](info): commencing operation
krb5kdc: starting...
W20260807 06:16:45.210095 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:45 dist-test-slave-h3ft krb5kdc[7802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:45.358726 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:45.493541 7820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:45.493971 7820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:45.494061 7820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:45.498443 7820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:45.498571 7820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:45.498616 7820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:45.498682 7820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:45.498725 7820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:45.504704 7820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:45.506179 7820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:45.507572 7820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:45.517989 7826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:45.518157 7820 server_base.cc:1034] running on GCE node
W20260807 06:16:45.517995 7831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:45.518357 7825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:45.519156 7820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:45.520399 7820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:45.521576 7820 hybrid_clock.cc:648] HybridClock initialized: now 1786083405521547 us; error 46 us; skew 500 ppm
Aug 07 06:16:45 dist-test-slave-h3ft krb5kdc[7802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083405, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:45.529659 7820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:45.530755 7820 webserver.cc:460] Webserver started at http://127.0.0.1:42581/ using document root <none> and password file <none>
I20260807 06:16:45.531551 7820 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:45.531656 7820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:45.531946 7820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:45.534547 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "baceba15f78b4c2fb39c8236db784283"
format_stamp: "Formatted at 2026-08-07 06:16:45 on dist-test-slave-h3ft"
server_key: "125f86053b37b2263416f1072a8e2097"
server_key_iv: "2eb4f834af08b884984d2668319bb8f1"
server_key_version: "encryptionkey@0"
I20260807 06:16:45.535334 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "baceba15f78b4c2fb39c8236db784283"
format_stamp: "Formatted at 2026-08-07 06:16:45 on dist-test-slave-h3ft"
server_key: "125f86053b37b2263416f1072a8e2097"
server_key_iv: "2eb4f834af08b884984d2668319bb8f1"
server_key_version: "encryptionkey@0"
I20260807 06:16:45.540079 7820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 06:16:45.543126 7839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:45.544739 7820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:16:45.544955 7820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "baceba15f78b4c2fb39c8236db784283"
format_stamp: "Formatted at 2026-08-07 06:16:45 on dist-test-slave-h3ft"
server_key: "125f86053b37b2263416f1072a8e2097"
server_key_iv: "2eb4f834af08b884984d2668319bb8f1"
server_key_version: "encryptionkey@0"
I20260807 06:16:45.545110 7820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:45.577080 7820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:45.581266 7820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:45.581529 7820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:45.590930 7820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36831
I20260807 06:16:45.590946 7907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36831 every 8 connection(s)
I20260807 06:16:45.592268 7820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:45.596065 7908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:45.598306 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7820
I20260807 06:16:45.598574 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:45.598927 9750 external_mini_cluster.cc:1428] Setting key 3875ac2f111d980c1e3cdb2d00a40abd
I20260807 06:16:45.604605 7908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283: Bootstrap starting.
I20260807 06:16:45.608095 7908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:45.609253 7908 log.cc:824] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:45.610944 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:45.600770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:56167 (local address 127.0.0.1:36831)
0807 06:16:45.601286 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:45.601308 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:45.601357 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:16:45.602296 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:45.602329 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:45.602865 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:45.603375 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:45.603854 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:45.605412 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:45.606426 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:45.607215 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:45.607384 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:45.608590 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:45.608597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:45.608616 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:45.609222 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:45.609315 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:45.609354 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:45.609497 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:45.609755 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 06:16:45.609936 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":143,"threads_started":1}
I20260807 06:16:45.612068 7908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283: No bootstrap required, opened a new log
I20260807 06:16:45.614606 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:45.616667 7908 raft_consensus.cc:348] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baceba15f78b4c2fb39c8236db784283" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36831 } }
I20260807 06:16:45.616981 7908 raft_consensus.cc:374] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:45.617074 7908 raft_consensus.cc:729] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baceba15f78b4c2fb39c8236db784283, State: Initialized, Role: FOLLOWER
I20260807 06:16:45.617646 7908 consensus_queue.cc:260] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baceba15f78b4c2fb39c8236db784283" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36831 } }
I20260807 06:16:45.617833 7908 raft_consensus.cc:388] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:45.617975 7908 raft_consensus.cc:482] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:45.618122 7908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:45.619516 7908 raft_consensus.cc:504] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baceba15f78b4c2fb39c8236db784283" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36831 } }
I20260807 06:16:45.619972 7908 leader_election.cc:302] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baceba15f78b4c2fb39c8236db784283; no voters:
I20260807 06:16:45.620376 7908 leader_election.cc:288] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:45.620496 7921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:45.620771 7921 raft_consensus.cc:686] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [term 1 LEADER]: Becoming Leader. State: Replica: baceba15f78b4c2fb39c8236db784283, State: Running, Role: LEADER
I20260807 06:16:45.621575 7921 consensus_queue.cc:237] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baceba15f78b4c2fb39c8236db784283" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36831 } }
I20260807 06:16:45.622113 7908 sys_catalog.cc:564] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:45.622246 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:45.617250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:50291 (local address 127.0.0.1:36831)
0807 06:16:45.617398 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:45.617410 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:45.617593 (+ 183us) server_negotiation.cc:407] Connection header received
0807 06:16:45.617697 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:45.617700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:45.617756 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:45.617850 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:45.618192 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:45.618978 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:45.619882 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:45.620289 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:45.620416 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:45.621505 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:45.621509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:45.621512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:45.621528 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:45.621561 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:45.621575 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:45.621672 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:45.622009 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 06:16:45.622057 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 06:16:45.623714 7924 sys_catalog.cc:455] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baceba15f78b4c2fb39c8236db784283. Latest consensus state: current_term: 1 leader_uuid: "baceba15f78b4c2fb39c8236db784283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baceba15f78b4c2fb39c8236db784283" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36831 } } }
I20260807 06:16:45.623843 7923 sys_catalog.cc:455] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baceba15f78b4c2fb39c8236db784283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baceba15f78b4c2fb39c8236db784283" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36831 } } }
I20260807 06:16:45.623850 7924 sys_catalog.cc:458] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:45.624066 7923 sys_catalog.cc:458] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283 [sys.catalog]: This master's current role is: LEADER
W20260807 06:16:45.628351 7939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:45.628440 7939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:45.628526 7934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:45.629338 7934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:45.635643 7934 catalog_manager.cc:1347] Generated new cluster ID: 53785731f1384f92815db9b4cf6b870a
I20260807 06:16:45.635730 7934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:45.640981 7934 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:45.642107 7934 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:45.651925 7934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P baceba15f78b4c2fb39c8236db784283: Generated new TSK 0
I20260807 06:16:45.653252 7934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:45.660879 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7820
W20260807 06:16:45.680990 7800 connection.cc:535] client connection to 127.0.0.1:36831 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36831): Connection reset by peer (error 104)
2026-08-07T06:16:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1780 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:45 dist-test-slave-h3ft krb5kdc[7945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:45 dist-test-slave-h3ft krb5kdc[7945](info): set up 2 sockets
Aug 07 06:16:45 dist-test-slave-h3ft krb5kdc[7945](info): commencing operation
krb5kdc: starting...
W20260807 06:16:46.945673 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:47 dist-test-slave-h3ft krb5kdc[7945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:47.103591 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:47.238430 7962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:47.238803 7962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:47.238888 7962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:47.243099 7962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:47.243202 7962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:47.243242 7962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:47.243294 7962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:47.243332 7962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:47.249009 7962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:47.250473 7962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:47.251866 7962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:47.262162 7973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:47.262166 7968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:47.262290 7967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:47.262584 7962 server_base.cc:1034] running on GCE node
I20260807 06:16:47.263358 7962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:47.264678 7962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:47.265882 7962 hybrid_clock.cc:648] HybridClock initialized: now 1786083407265854 us; error 32 us; skew 500 ppm
Aug 07 06:16:47 dist-test-slave-h3ft krb5kdc[7945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083407, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:47.269248 7962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:47.270313 7962 webserver.cc:460] Webserver started at http://127.0.0.1:46253/ using document root <none> and password file <none>
I20260807 06:16:47.270953 7962 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:47.271077 7962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:47.271310 7962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:47.273752 7962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "46bb05e2da1742cbba3b705d79a45f3c"
format_stamp: "Formatted at 2026-08-07 06:16:47 on dist-test-slave-h3ft"
server_key: "156c3d7dcfd9824980603ba42c53f297"
server_key_iv: "1d5eb246f57e46fca0ec050f10b0e903"
server_key_version: "encryptionkey@0"
I20260807 06:16:47.274376 7962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "46bb05e2da1742cbba3b705d79a45f3c"
format_stamp: "Formatted at 2026-08-07 06:16:47 on dist-test-slave-h3ft"
server_key: "156c3d7dcfd9824980603ba42c53f297"
server_key_iv: "1d5eb246f57e46fca0ec050f10b0e903"
server_key_version: "encryptionkey@0"
I20260807 06:16:47.278728 7962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 06:16:47.281718 7991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:47.283294 7962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:47.283416 7962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "46bb05e2da1742cbba3b705d79a45f3c"
format_stamp: "Formatted at 2026-08-07 06:16:47 on dist-test-slave-h3ft"
server_key: "156c3d7dcfd9824980603ba42c53f297"
server_key_iv: "1d5eb246f57e46fca0ec050f10b0e903"
server_key_version: "encryptionkey@0"
I20260807 06:16:47.283519 7962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:47.288815 7962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:47.292385 7962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:47.292551 7962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:47.301570 7962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46227
I20260807 06:16:47.301614 8066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46227 every 8 connection(s)
I20260807 06:16:47.302786 7962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:47.303287 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 7962
I20260807 06:16:47.303472 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:47.303853 9750 external_mini_cluster.cc:1428] Setting key 3f461757e5f3a863aa4a118e0679d8bd
I20260807 06:16:47.306948 8067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:47.310314 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:47.305600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:49601 (local address 127.0.0.1:46227)
0807 06:16:47.306103 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:47.306124 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:47.306177 (+ 53us) server_negotiation.cc:407] Connection header received
0807 06:16:47.307117 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:47.307149 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:47.307508 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:47.307847 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:47.308866 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:47.308872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:47.308885 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:47.308896 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:47.308959 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:47.308981 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:47.309110 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:47.309233 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 06:16:47.309440 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":158,"threads_started":1}
I20260807 06:16:47.313639 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:47.314630 8067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c: Bootstrap starting.
I20260807 06:16:47.317819 8067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:47.318152 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:47.316164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:51513 (local address 127.0.0.1:46227)
0807 06:16:47.316323 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:47.316328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:47.316619 (+ 291us) server_negotiation.cc:407] Connection header received
0807 06:16:47.316760 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:47.316762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:47.316798 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:47.316859 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:47.317651 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:47.317654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:47.317656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:47.317667 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:47.317696 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:47.317707 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:47.317778 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:47.317935 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 06:16:47.317987 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:16:47.318866 8067 log.cc:824] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:47.321184 8067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c: No bootstrap required, opened a new log
I20260807 06:16:47.325289 8067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46bb05e2da1742cbba3b705d79a45f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260807 06:16:47.325564 8067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:47.325644 8067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46bb05e2da1742cbba3b705d79a45f3c, State: Initialized, Role: FOLLOWER
I20260807 06:16:47.326141 8067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46bb05e2da1742cbba3b705d79a45f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260807 06:16:47.326303 8067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:47.326386 8067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:47.326516 8067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:47.327677 8067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46bb05e2da1742cbba3b705d79a45f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260807 06:16:47.328066 8067 leader_election.cc:302] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46bb05e2da1742cbba3b705d79a45f3c; no voters:
I20260807 06:16:47.328437 8067 leader_election.cc:288] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:47.328729 8081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:47.329345 8081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [term 1 LEADER]: Becoming Leader. State: Replica: 46bb05e2da1742cbba3b705d79a45f3c, State: Running, Role: LEADER
I20260807 06:16:47.329758 8081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46bb05e2da1742cbba3b705d79a45f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260807 06:16:47.330078 8067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:47.331779 8083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46bb05e2da1742cbba3b705d79a45f3c. Latest consensus state: current_term: 1 leader_uuid: "46bb05e2da1742cbba3b705d79a45f3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46bb05e2da1742cbba3b705d79a45f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } } }
I20260807 06:16:47.331949 8083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:47.332345 8082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46bb05e2da1742cbba3b705d79a45f3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46bb05e2da1742cbba3b705d79a45f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } } }
I20260807 06:16:47.332435 8082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c [sys.catalog]: This master's current role is: LEADER
W20260807 06:16:47.335947 8099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:47.336030 8099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:47.336118 8098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:47.336875 8098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:47.343581 8098 catalog_manager.cc:1347] Generated new cluster ID: ea2169e0cd9046fbb7dde5e78fd2f456
I20260807 06:16:47.343657 8098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:47.360666 8098 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:47.361167 8098 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:47.371946 8098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46bb05e2da1742cbba3b705d79a45f3c: Generated new TSK 0
I20260807 06:16:47.373073 8098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:47.379678 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 7962
2026-08-07T06:16:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1714 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:47 dist-test-slave-h3ft krb5kdc[8108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:47 dist-test-slave-h3ft krb5kdc[8108](info): set up 2 sockets
Aug 07 06:16:47 dist-test-slave-h3ft krb5kdc[8108](info): commencing operation
krb5kdc: starting...
W20260807 06:16:48.647528 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:48 dist-test-slave-h3ft krb5kdc[8108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:48.806502 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:48.945786 8133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:48.946192 8133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:48.946278 8133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:48.950654 8133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:48.950785 8133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:48.950824 8133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:48.950866 8133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:48.950904 8133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:48.957023 8133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:48.958554 8133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:48.960037 8133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:48.970788 8147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:48.970835 8139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:48.970921 8133 server_base.cc:1034] running on GCE node
W20260807 06:16:48.971105 8140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:48.971966 8133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:48.973343 8133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:48.974594 8133 hybrid_clock.cc:648] HybridClock initialized: now 1786083408974574 us; error 30 us; skew 500 ppm
Aug 07 06:16:48 dist-test-slave-h3ft krb5kdc[8108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083408, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:48.981849 8133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:48.983536 8133 webserver.cc:460] Webserver started at http://127.0.0.1:37349/ using document root <none> and password file <none>
I20260807 06:16:48.984468 8133 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:48.984592 8133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:48.984952 8133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:48.988881 8133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "d70b553eb1f84c7ba700a9be5efc0aca"
format_stamp: "Formatted at 2026-08-07 06:16:48 on dist-test-slave-h3ft"
server_key: "7f4534d3ca07069e00efcb796a779b96"
server_key_iv: "8f7138fb2219b9f688186fe8e6a1553f"
server_key_version: "encryptionkey@0"
I20260807 06:16:48.989843 8133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "d70b553eb1f84c7ba700a9be5efc0aca"
format_stamp: "Formatted at 2026-08-07 06:16:48 on dist-test-slave-h3ft"
server_key: "7f4534d3ca07069e00efcb796a779b96"
server_key_iv: "8f7138fb2219b9f688186fe8e6a1553f"
server_key_version: "encryptionkey@0"
I20260807 06:16:48.997411 8133 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 06:16:49.002275 8158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:49.004741 8133 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:49.004966 8133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "d70b553eb1f84c7ba700a9be5efc0aca"
format_stamp: "Formatted at 2026-08-07 06:16:48 on dist-test-slave-h3ft"
server_key: "7f4534d3ca07069e00efcb796a779b96"
server_key_iv: "8f7138fb2219b9f688186fe8e6a1553f"
server_key_version: "encryptionkey@0"
I20260807 06:16:49.005188 8133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:49.030217 8133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:49.035915 8133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:49.036222 8133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:49.046842 8133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40431
I20260807 06:16:49.046892 8228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40431 every 8 connection(s)
I20260807 06:16:49.048367 8133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:49.049281 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 8133
I20260807 06:16:49.049489 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:49.049983 9750 external_mini_cluster.cc:1428] Setting key 556f1ef9e02d2cb42ac5e153405db1bc
I20260807 06:16:49.053153 8229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 06:16:49 dist-test-slave-h3ft krb5kdc[8108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 06:16:49.061286 8229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca: Bootstrap starting.
I20260807 06:16:49.064826 8229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:49.065994 8229 log.cc:824] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:49.068111 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:49.051914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:56303 (local address 127.0.0.1:40431)
0807 06:16:49.052531 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:49.052557 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:49.052633 (+ 76us) server_negotiation.cc:407] Connection header received
0807 06:16:49.053640 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:49.053666 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:49.054109 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:49.054531 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:49.055650 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:49.056834 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:49.057801 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:49.058361 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:49.058564 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:49.061979 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:49.061986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:49.062004 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:49.062040 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:49.064090 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:49.064799 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:49.064805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:49.064811 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:49.065067 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:49.065495 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:49.065498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:49.065499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:49.066033 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:49.066296 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:49.066719 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 06:16:49.067102 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":202,"threads_started":1}
I20260807 06:16:49.068783 8229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca: No bootstrap required, opened a new log
I20260807 06:16:49.071975 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:49.074153 8229 raft_consensus.cc:348] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260807 06:16:49.074592 8229 raft_consensus.cc:374] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:49.074695 8229 raft_consensus.cc:729] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d70b553eb1f84c7ba700a9be5efc0aca, State: Initialized, Role: FOLLOWER
I20260807 06:16:49.075388 8229 consensus_queue.cc:260] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260807 06:16:49.075623 8229 raft_consensus.cc:388] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:49.075707 8229 raft_consensus.cc:482] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:49.075834 8229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:49.077278 8229 raft_consensus.cc:504] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260807 06:16:49.077749 8229 leader_election.cc:302] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d70b553eb1f84c7ba700a9be5efc0aca; no voters:
I20260807 06:16:49.078167 8229 leader_election.cc:288] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:49.078256 8242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:49.078516 8242 raft_consensus.cc:686] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [term 1 LEADER]: Becoming Leader. State: Replica: d70b553eb1f84c7ba700a9be5efc0aca, State: Running, Role: LEADER
I20260807 06:16:49.078903 8242 consensus_queue.cc:237] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260807 06:16:49.080005 8229 sys_catalog.cc:564] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:49.082528 8243 sys_catalog.cc:455] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } } }
I20260807 06:16:49.082916 8243 sys_catalog.cc:458] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:49.083316 8245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader d70b553eb1f84c7ba700a9be5efc0aca. Latest consensus state: current_term: 1 leader_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d70b553eb1f84c7ba700a9be5efc0aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } } }
I20260807 06:16:49.083458 8245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:49.086457 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:49.075094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:52239 (local address 127.0.0.1:40431)
0807 06:16:49.075253 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:49.075259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:49.075482 (+ 223us) server_negotiation.cc:407] Connection header received
0807 06:16:49.075657 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:49.075660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:49.075730 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:49.075834 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:49.076624 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:49.077460 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:49.078768 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:49.079242 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:49.079355 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:49.081815 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:49.081823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:49.081829 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:49.081866 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:49.084160 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:49.084924 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:49.084927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:49.084928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:49.085017 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:49.085428 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:49.085431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:49.085433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:49.085667 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:49.085848 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:49.086156 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 06:16:49.086271 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 06:16:49.087070 8258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:49.087177 8258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:49.087284 8260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:49.088485 8260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:49.096431 8260 catalog_manager.cc:1347] Generated new cluster ID: 3b9bfb3bfc3f4cffabf5e6f3098f39b9
I20260807 06:16:49.096563 8260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:49.109356 8260 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:49.110914 8260 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:49.118829 8260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d70b553eb1f84c7ba700a9be5efc0aca: Generated new TSK 0
I20260807 06:16:49.119867 8260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:49.124660 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 8133
W20260807 06:16:49.138695 8105 connection.cc:535] client connection to 127.0.0.1:40431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40431): Connection reset by peer (error 104)
2026-08-07T06:16:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1743 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:49 dist-test-slave-h3ft krb5kdc[8267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:49 dist-test-slave-h3ft krb5kdc[8267](info): set up 2 sockets
Aug 07 06:16:49 dist-test-slave-h3ft krb5kdc[8267](info): commencing operation
krb5kdc: starting...
W20260807 06:16:50.410058 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:50 dist-test-slave-h3ft krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:50.559258 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:50.699435 8283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:50.699811 8283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:50.699860 8283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:50.704156 8283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:50.704242 8283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:50.704262 8283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:50.704279 8283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:50.704296 8283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:50.710424 8283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:50.711809 8283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:50.713097 8283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:50.723361 8289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:50.723369 8292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:50.723398 8290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:50.723932 8283 server_base.cc:1034] running on GCE node
I20260807 06:16:50.724663 8283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:50.726065 8283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:50.727264 8283 hybrid_clock.cc:648] HybridClock initialized: now 1786083410727228 us; error 47 us; skew 500 ppm
Aug 07 06:16:50 dist-test-slave-h3ft krb5kdc[8267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:50.735826 8283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:50.737043 8283 webserver.cc:460] Webserver started at http://127.0.0.1:34371/ using document root <none> and password file <none>
I20260807 06:16:50.737775 8283 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:50.737877 8283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:50.738166 8283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:50.740780 8283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "db26abf97c4b4cb8813ed4ec0250b9f4"
format_stamp: "Formatted at 2026-08-07 06:16:50 on dist-test-slave-h3ft"
server_key: "7f1b6a904fb3298e1caf2d477c57236e"
server_key_iv: "71819fc07af456c679260568989f0539"
server_key_version: "encryptionkey@0"
I20260807 06:16:50.741442 8283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "db26abf97c4b4cb8813ed4ec0250b9f4"
format_stamp: "Formatted at 2026-08-07 06:16:50 on dist-test-slave-h3ft"
server_key: "7f1b6a904fb3298e1caf2d477c57236e"
server_key_iv: "71819fc07af456c679260568989f0539"
server_key_version: "encryptionkey@0"
I20260807 06:16:50.745855 8283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 06:16:50.748685 8299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:50.750111 8283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:16:50.750268 8283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "db26abf97c4b4cb8813ed4ec0250b9f4"
format_stamp: "Formatted at 2026-08-07 06:16:50 on dist-test-slave-h3ft"
server_key: "7f1b6a904fb3298e1caf2d477c57236e"
server_key_iv: "71819fc07af456c679260568989f0539"
server_key_version: "encryptionkey@0"
I20260807 06:16:50.750389 8283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:50.762804 8283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:50.766420 8283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:50.766655 8283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:50.775540 8283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42717
I20260807 06:16:50.775588 8369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42717 every 8 connection(s)
I20260807 06:16:50.776710 8283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:50.777925 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 8283
I20260807 06:16:50.778103 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:50.778461 9750 external_mini_cluster.cc:1428] Setting key 553140ba659903a43685076d567d0944
I20260807 06:16:50.780690 8370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:50.789314 8370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4: Bootstrap starting.
I20260807 06:16:50.790130 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:50.780189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:48925 (local address 127.0.0.1:42717)
0807 06:16:50.780681 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:50.780707 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:50.780763 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:16:50.781772 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:50.781800 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:50.782298 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:50.782738 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:50.783231 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:50.784879 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:50.785861 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:50.786583 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:50.786766 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:50.787858 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:50.787866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:50.787885 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:50.788478 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:50.788556 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:50.788589 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:50.788731 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:50.788867 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 06:16:50.789025 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":146,"threads_started":1}
I20260807 06:16:50.792567 8370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:50.793628 8370 log.cc:824] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:50.793815 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:50.796209 8370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4: No bootstrap required, opened a new log
I20260807 06:16:50.801226 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:50.796500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:40375 (local address 127.0.0.1:42717)
0807 06:16:50.796643 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:50.796648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:50.796890 (+ 242us) server_negotiation.cc:407] Connection header received
0807 06:16:50.797030 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:50.797033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:50.797076 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:50.797144 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:50.797474 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:50.797989 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:50.799274 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:50.799499 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:50.799597 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:50.800608 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:50.800611 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:50.800614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:50.800627 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:50.800655 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:50.800665 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:50.800739 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:50.800999 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 06:16:50.801048 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:16:50.801589 8370 raft_consensus.cc:348] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42717 } }
I20260807 06:16:50.801955 8370 raft_consensus.cc:374] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:50.802016 8370 raft_consensus.cc:729] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db26abf97c4b4cb8813ed4ec0250b9f4, State: Initialized, Role: FOLLOWER
I20260807 06:16:50.802600 8370 consensus_queue.cc:260] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42717 } }
I20260807 06:16:50.802783 8370 raft_consensus.cc:388] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:50.802836 8370 raft_consensus.cc:482] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:50.802930 8370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:50.804111 8370 raft_consensus.cc:504] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42717 } }
I20260807 06:16:50.804467 8370 leader_election.cc:302] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db26abf97c4b4cb8813ed4ec0250b9f4; no voters:
I20260807 06:16:50.804764 8370 leader_election.cc:288] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:50.804916 8384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:50.805179 8384 raft_consensus.cc:686] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [term 1 LEADER]: Becoming Leader. State: Replica: db26abf97c4b4cb8813ed4ec0250b9f4, State: Running, Role: LEADER
I20260807 06:16:50.805593 8384 consensus_queue.cc:237] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42717 } }
I20260807 06:16:50.806393 8370 sys_catalog.cc:564] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:50.806907 8384 sys_catalog.cc:455] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db26abf97c4b4cb8813ed4ec0250b9f4. Latest consensus state: current_term: 1 leader_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42717 } } }
I20260807 06:16:50.807076 8384 sys_catalog.cc:458] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:50.807469 8388 sys_catalog.cc:455] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db26abf97c4b4cb8813ed4ec0250b9f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42717 } } }
I20260807 06:16:50.807607 8388 sys_catalog.cc:458] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:50.807529 8390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:50.811764 8390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:50.818807 8390 catalog_manager.cc:1347] Generated new cluster ID: fc35c73fd7f94014a9447d331bfa6918
I20260807 06:16:50.818907 8390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:50.831480 8390 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:50.832702 8390 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:50.837241 8390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db26abf97c4b4cb8813ed4ec0250b9f4: Generated new TSK 0
I20260807 06:16:50.838085 8390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:50.845297 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 8283
W20260807 06:16:50.863298 8265 connection.cc:535] client connection to 127.0.0.1:42717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42717): Connection reset by peer (error 104)
2026-08-07T06:16:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1724 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:50 dist-test-slave-h3ft krb5kdc[8411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:50 dist-test-slave-h3ft krb5kdc[8411](info): set up 2 sockets
Aug 07 06:16:50 dist-test-slave-h3ft krb5kdc[8411](info): commencing operation
krb5kdc: starting...
W20260807 06:16:52.104419 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:52 dist-test-slave-h3ft krb5kdc[8411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 06:16:52.258579 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43603
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:52.398561 8431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:52.398985 8431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:52.399096 8431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:52.404016 8431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:52.404135 8431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:52.404175 8431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:52.404215 8431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:52.404253 8431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:52.410477 8431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:52.412086 8431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:52.413496 8431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:52.424000 8438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:52.424151 8440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:52.424031 8437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:52.424461 8431 server_base.cc:1034] running on GCE node
I20260807 06:16:52.425450 8431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:52.427083 8431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:52.428316 8431 hybrid_clock.cc:648] HybridClock initialized: now 1786083412428295 us; error 45 us; skew 500 ppm
Aug 07 06:16:52 dist-test-slave-h3ft krb5kdc[8411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083412, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:52.432116 8431 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 06:16:52.433346 8431 webserver.cc:460] Webserver started at http://127.0.0.1:42411/ using document root <none> and password file <none>
I20260807 06:16:52.434072 8431 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:52.434175 8431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:52.434465 8431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:52.437098 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "c382abf0c4764b088fe303430cc36cc3"
format_stamp: "Formatted at 2026-08-07 06:16:52 on dist-test-slave-h3ft"
server_key: "19902f56f7fdea847bdd7fbc951d2a7f"
server_key_iv: "2a7ee6e074e54b8c7d285bcabdcf4d90"
server_key_version: "encryptionkey@0"
I20260807 06:16:52.437768 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "c382abf0c4764b088fe303430cc36cc3"
format_stamp: "Formatted at 2026-08-07 06:16:52 on dist-test-slave-h3ft"
server_key: "19902f56f7fdea847bdd7fbc951d2a7f"
server_key_iv: "2a7ee6e074e54b8c7d285bcabdcf4d90"
server_key_version: "encryptionkey@0"
I20260807 06:16:52.442454 8431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 06:16:52.445546 8451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:52.447284 8431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:16:52.447451 8431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "c382abf0c4764b088fe303430cc36cc3"
format_stamp: "Formatted at 2026-08-07 06:16:52 on dist-test-slave-h3ft"
server_key: "19902f56f7fdea847bdd7fbc951d2a7f"
server_key_iv: "2a7ee6e074e54b8c7d285bcabdcf4d90"
server_key_version: "encryptionkey@0"
I20260807 06:16:52.447592 8431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:52.453054 8431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:52.456830 8431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:52.457064 8431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:52.465678 8431 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43603
I20260807 06:16:52.465699 8522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43603 every 8 connection(s)
I20260807 06:16:52.466890 8431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:52.468734 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 8431
I20260807 06:16:52.468928 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:52.469304 9750 external_mini_cluster.cc:1428] Setting key 33ba057cddd7c0ae51f75596bf370055
I20260807 06:16:52.471179 8523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:52.475754 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:52.470951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:46673 (local address 127.0.0.1:43603)
0807 06:16:52.471547 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:52.471566 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:52.471611 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:16:52.472386 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:52.472405 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:52.472765 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:52.473120 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:52.474188 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:52.474193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:52.474207 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:52.474218 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:52.474284 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:52.474306 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:52.474434 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:52.474576 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 06:16:52.474782 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":182,"threads_started":1}
I20260807 06:16:52.479089 8523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3: Bootstrap starting.
I20260807 06:16:52.479187 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:52.482079 8523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:52.483237 8523 log.cc:824] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:52.483506 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:52.481714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:58413 (local address 127.0.0.1:43603)
0807 06:16:52.481871 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:52.481877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:52.482073 (+ 196us) server_negotiation.cc:407] Connection header received
0807 06:16:52.482161 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:52.482165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:52.482207 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:52.482290 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:52.483083 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:52.483086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:52.483089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:52.483102 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:52.483134 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:52.483147 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:52.483244 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:52.483327 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 06:16:52.483385 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:16:52.485739 8523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3: No bootstrap required, opened a new log
I20260807 06:16:52.489919 8523 raft_consensus.cc:348] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c382abf0c4764b088fe303430cc36cc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43603 } }
I20260807 06:16:52.490176 8523 raft_consensus.cc:374] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:52.490268 8523 raft_consensus.cc:729] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c382abf0c4764b088fe303430cc36cc3, State: Initialized, Role: FOLLOWER
I20260807 06:16:52.490811 8523 consensus_queue.cc:260] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c382abf0c4764b088fe303430cc36cc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43603 } }
I20260807 06:16:52.490981 8523 raft_consensus.cc:388] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:52.491074 8523 raft_consensus.cc:482] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:52.491217 8523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:52.492244 8523 raft_consensus.cc:504] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c382abf0c4764b088fe303430cc36cc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43603 } }
I20260807 06:16:52.492661 8523 leader_election.cc:302] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c382abf0c4764b088fe303430cc36cc3; no voters:
I20260807 06:16:52.493016 8523 leader_election.cc:288] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:52.493160 8535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:52.493438 8535 raft_consensus.cc:686] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [term 1 LEADER]: Becoming Leader. State: Replica: c382abf0c4764b088fe303430cc36cc3, State: Running, Role: LEADER
I20260807 06:16:52.493788 8535 consensus_queue.cc:237] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c382abf0c4764b088fe303430cc36cc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43603 } }
I20260807 06:16:52.494534 8523 sys_catalog.cc:564] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:52.495973 8538 sys_catalog.cc:455] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c382abf0c4764b088fe303430cc36cc3. Latest consensus state: current_term: 1 leader_uuid: "c382abf0c4764b088fe303430cc36cc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c382abf0c4764b088fe303430cc36cc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43603 } } }
I20260807 06:16:52.495994 8537 sys_catalog.cc:455] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c382abf0c4764b088fe303430cc36cc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c382abf0c4764b088fe303430cc36cc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43603 } } }
I20260807 06:16:52.496125 8537 sys_catalog.cc:458] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:52.496126 8538 sys_catalog.cc:458] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:52.497295 8550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:52.501183 8550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:52.507655 8550 catalog_manager.cc:1347] Generated new cluster ID: 33f79461690f48e1b136bf6e59290c5f
I20260807 06:16:52.507745 8550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:52.514829 8550 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:52.515260 8550 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:52.524827 8550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c382abf0c4764b088fe303430cc36cc3: Generated new TSK 0
I20260807 06:16:52.525904 8550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:52.533826 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 8431
2026-08-07T06:16:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1690 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:52 dist-test-slave-h3ft krb5kdc[8565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:52 dist-test-slave-h3ft krb5kdc[8565](info): set up 2 sockets
Aug 07 06:16:52 dist-test-slave-h3ft krb5kdc[8565](info): commencing operation
krb5kdc: starting...
W20260807 06:16:53.818965 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:53 dist-test-slave-h3ft krb5kdc[8565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:53.979666 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42825
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:54.118228 8585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:54.118637 8585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:54.118723 8585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:54.123076 8585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:54.123188 8585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:54.123237 8585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:54.123296 8585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:54.123335 8585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:54.129534 8585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42825
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:54.131102 8585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:54.132478 8585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:54.143123 8591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:54.143112 8595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:54.143112 8592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:54.143515 8585 server_base.cc:1034] running on GCE node
I20260807 06:16:54.144307 8585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:54.145829 8585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:54.147084 8585 hybrid_clock.cc:648] HybridClock initialized: now 1786083414147070 us; error 33 us; skew 500 ppm
Aug 07 06:16:54 dist-test-slave-h3ft krb5kdc[8565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083414, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:54.150628 8585 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:54.151868 8585 webserver.cc:460] Webserver started at http://127.9.133.190:42629/ using document root <none> and password file <none>
I20260807 06:16:54.152588 8585 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:54.152688 8585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:54.152981 8585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:54.155579 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad"
format_stamp: "Formatted at 2026-08-07 06:16:54 on dist-test-slave-h3ft"
server_key: "3b2662441ce37acffd6b7b22116fb045"
server_key_iv: "823294e12a297519c3d769cb4e270fff"
server_key_version: "encryptionkey@0"
I20260807 06:16:54.156250 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad"
format_stamp: "Formatted at 2026-08-07 06:16:54 on dist-test-slave-h3ft"
server_key: "3b2662441ce37acffd6b7b22116fb045"
server_key_iv: "823294e12a297519c3d769cb4e270fff"
server_key_version: "encryptionkey@0"
I20260807 06:16:54.160967 8585 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 06:16:54.164014 8605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:54.165500 8585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:16:54.165665 8585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad"
format_stamp: "Formatted at 2026-08-07 06:16:54 on dist-test-slave-h3ft"
server_key: "3b2662441ce37acffd6b7b22116fb045"
server_key_iv: "823294e12a297519c3d769cb4e270fff"
server_key_version: "encryptionkey@0"
I20260807 06:16:54.165802 8585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:54.180203 8585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:54.184309 8585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:54.184602 8585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:54.194514 8585 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:42825
I20260807 06:16:54.194550 8672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:42825 every 8 connection(s)
I20260807 06:16:54.195815 8585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:54.199481 8673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:54.199808 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 8585
I20260807 06:16:54.199973 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:54.200340 9750 external_mini_cluster.cc:1428] Setting key 110c486e36c950e5d74151083b459a6f
I20260807 06:16:54.208534 8673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad: Bootstrap starting.
Aug 07 06:16:54 dist-test-slave-h3ft krb5kdc[8565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:54.211988 8673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:54.213114 8673 log.cc:824] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:54.215767 8673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad: No bootstrap required, opened a new log
I20260807 06:16:54.217845 8679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:54.202186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49465 (local address 127.9.133.190:42825)
0807 06:16:54.202696 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:54.202719 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:54.202775 (+ 56us) server_negotiation.cc:407] Connection header received
0807 06:16:54.203702 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:54.203733 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:54.204286 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:54.204739 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:54.205575 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:54.207283 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:54.208221 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:54.208951 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:54.209128 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:54.212333 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:54.212339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:54.212357 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:54.212391 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:54.214412 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:54.215065 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:54.215073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:54.215078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:54.215207 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:54.215590 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:54.215599 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:54.215600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:54.216119 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:54.216357 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:54.216713 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 06:16:54.216980 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":135,"threads_started":1}
I20260807 06:16:54.220242 8673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42825 } }
I20260807 06:16:54.220605 8673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:54.220695 8673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f75a8d74b1e42adb7115ae6ccf3cbad, State: Initialized, Role: FOLLOWER
I20260807 06:16:54.221215 8673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42825 } }
I20260807 06:16:54.221413 8673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:54.221454 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:54.221516 8673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:54.221640 8673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:54.222764 8673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42825 } }
I20260807 06:16:54.223237 8673 leader_election.cc:302] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f75a8d74b1e42adb7115ae6ccf3cbad; no voters:
I20260807 06:16:54.223594 8673 leader_election.cc:288] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:54.223716 8686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:54.223961 8686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [term 1 LEADER]: Becoming Leader. State: Replica: 0f75a8d74b1e42adb7115ae6ccf3cbad, State: Running, Role: LEADER
I20260807 06:16:54.224529 8686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42825 } }
I20260807 06:16:54.225208 8673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:54.226671 8688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f75a8d74b1e42adb7115ae6ccf3cbad. Latest consensus state: current_term: 1 leader_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42825 } } }
I20260807 06:16:54.226756 8687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f75a8d74b1e42adb7115ae6ccf3cbad" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42825 } } }
I20260807 06:16:54.226859 8688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:54.226883 8687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:54.227845 8699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:54.232972 8699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:54.235952 8679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:54.224149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42007 (local address 127.9.133.190:42825)
0807 06:16:54.224259 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:54.224264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:54.226178 (+ 1914us) server_negotiation.cc:407] Connection header received
0807 06:16:54.226410 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:54.226413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:54.226476 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:54.226574 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:54.227600 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:54.228400 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:54.229498 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:54.229739 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:54.229824 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:54.231703 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:54.231707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:54.231710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:54.231742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:54.233705 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:54.234325 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:54.234329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:54.234331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:54.234444 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:54.234846 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:54.234849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:54.234851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:54.235131 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:54.235297 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:54.235647 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 06:16:54.235766 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 06:16:54.240589 8699 catalog_manager.cc:1347] Generated new cluster ID: cd60a7e892eb4b63ab946d293cca0987
I20260807 06:16:54.240666 8699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:54.248597 8699 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:54.250102 8699 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:54.269634 8699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f75a8d74b1e42adb7115ae6ccf3cbad: Generated new TSK 0
I20260807 06:16:54.270730 8699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:54.276873 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 8585
W20260807 06:16:54.291767 8562 connection.cc:535] client connection to 127.9.133.190:42825 recv error: Network error: failed to read from TLS socket (remote: 127.9.133.190:42825): Connection reset by peer (error 104)
2026-08-07T06:16:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1738 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:54 dist-test-slave-h3ft krb5kdc[8715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:54 dist-test-slave-h3ft krb5kdc[8715](info): set up 2 sockets
Aug 07 06:16:54 dist-test-slave-h3ft krb5kdc[8715](info): commencing operation
krb5kdc: starting...
W20260807 06:16:55.557924 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:55 dist-test-slave-h3ft krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:55.719601 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45071
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:55.857800 8732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:55.858201 8732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:55.858276 8732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:55.862452 8732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:55.862561 8732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:55.862598 8732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:55.862639 8732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:55.862676 8732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:55.868355 8732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:45071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:55.869760 8732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:55.871124 8732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:55.882256 8739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:55.882256 8738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:55.882426 8732 server_base.cc:1034] running on GCE node
W20260807 06:16:55.882282 8743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:55.883450 8732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:55.884864 8732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:55.886107 8732 hybrid_clock.cc:648] HybridClock initialized: now 1786083415886072 us; error 46 us; skew 500 ppm
Aug 07 06:16:55 dist-test-slave-h3ft krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083415, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:55.894285 8732 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:55.895640 8732 webserver.cc:460] Webserver started at http://127.9.133.190:34015/ using document root <none> and password file <none>
I20260807 06:16:55.896374 8732 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:55.896487 8732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:55.896787 8732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:55.899511 8732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "11865862fac147db83d03b4ac5e69235"
format_stamp: "Formatted at 2026-08-07 06:16:55 on dist-test-slave-h3ft"
server_key: "d2d680379ae4e9fc4b83b3ddbfb7dcd7"
server_key_iv: "c009cad38052a48ff9d80c99a99bd6ae"
server_key_version: "encryptionkey@0"
I20260807 06:16:55.900211 8732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "11865862fac147db83d03b4ac5e69235"
format_stamp: "Formatted at 2026-08-07 06:16:55 on dist-test-slave-h3ft"
server_key: "d2d680379ae4e9fc4b83b3ddbfb7dcd7"
server_key_iv: "c009cad38052a48ff9d80c99a99bd6ae"
server_key_version: "encryptionkey@0"
I20260807 06:16:55.904861 8732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:16:55.907910 8752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:55.909461 8732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 06:16:55.909626 8732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "11865862fac147db83d03b4ac5e69235"
format_stamp: "Formatted at 2026-08-07 06:16:55 on dist-test-slave-h3ft"
server_key: "d2d680379ae4e9fc4b83b3ddbfb7dcd7"
server_key_iv: "c009cad38052a48ff9d80c99a99bd6ae"
server_key_version: "encryptionkey@0"
I20260807 06:16:55.909759 8732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:55.944700 8732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:55.949247 8732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:55.949535 8732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:55.959389 8732 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:45071
I20260807 06:16:55.959455 8830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:45071 every 8 connection(s)
I20260807 06:16:55.960696 8732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:55.964573 8831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:55.970780 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 8732
I20260807 06:16:55.970988 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:55.971416 9750 external_mini_cluster.cc:1428] Setting key f8fcaa1db0cec3d661a999f7959df6fd
I20260807 06:16:55.972481 8831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235: Bootstrap starting.
I20260807 06:16:55.975780 8831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:55.976768 8831 log.cc:824] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:55.979352 8831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235: No bootstrap required, opened a new log
I20260807 06:16:55.982775 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:55.973227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37375 (local address 127.9.133.190:45071)
0807 06:16:55.973743 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:55.973764 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:55.973808 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:16:55.974594 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:55.974616 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:55.975101 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:55.975458 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:55.975982 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:55.977165 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:55.978205 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:55.978754 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:55.978952 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:55.980186 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:55.980192 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:55.980209 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:55.980754 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:55.980824 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:55.980851 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:55.981011 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:55.981237 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 06:16:55.981413 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":171,"threads_started":1}
I20260807 06:16:55.983392 8831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11865862fac147db83d03b4ac5e69235" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45071 } }
I20260807 06:16:55.983676 8831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:55.983752 8831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11865862fac147db83d03b4ac5e69235, State: Initialized, Role: FOLLOWER
I20260807 06:16:55.984436 8831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11865862fac147db83d03b4ac5e69235" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45071 } }
I20260807 06:16:55.984668 8831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:55.984763 8831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:55.984905 8831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:55.986305 8831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11865862fac147db83d03b4ac5e69235" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45071 } }
I20260807 06:16:55.986565 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:55.986788 8831 leader_election.cc:302] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11865862fac147db83d03b4ac5e69235; no voters:
I20260807 06:16:55.987252 8831 leader_election.cc:288] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:55.987334 8837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:55.987582 8837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [term 1 LEADER]: Becoming Leader. State: Replica: 11865862fac147db83d03b4ac5e69235, State: Running, Role: LEADER
I20260807 06:16:55.987967 8837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11865862fac147db83d03b4ac5e69235" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45071 } }
I20260807 06:16:55.988914 8831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:55.990299 8839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11865862fac147db83d03b4ac5e69235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11865862fac147db83d03b4ac5e69235" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45071 } } }
I20260807 06:16:55.990456 8839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:55.990814 8838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11865862fac147db83d03b4ac5e69235. Latest consensus state: current_term: 1 leader_uuid: "11865862fac147db83d03b4ac5e69235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11865862fac147db83d03b4ac5e69235" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 45071 } } }
I20260807 06:16:55.990913 8838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:55.991122 8854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:55.994822 8854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:55.997524 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:55.989377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48629 (local address 127.9.133.190:45071)
0807 06:16:55.989526 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:55.989531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:55.992025 (+ 2494us) server_negotiation.cc:407] Connection header received
0807 06:16:55.992195 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:55.992197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:55.992245 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:55.992351 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:55.992713 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:55.993496 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:55.994785 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:55.995178 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:55.995279 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:55.996594 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:55.996599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:55.996602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:55.996619 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:55.996654 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:55.996669 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:55.996759 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:55.997310 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 06:16:55.997359 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:16:56.002964 8854 catalog_manager.cc:1347] Generated new cluster ID: ba0a300115d34175a2a5239593f554f3
I20260807 06:16:56.003161 8854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:56.025454 8854 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:56.027211 8854 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:56.037357 8854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11865862fac147db83d03b4ac5e69235: Generated new TSK 0
I20260807 06:16:56.038620 8854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:56.044184 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 8732
W20260807 06:16:56.065390 8713 connection.cc:535] client connection to 127.9.133.190:45071 recv error: Network error: failed to read from TLS socket (remote: 127.9.133.190:45071): Connection reset by peer (error 104)
2026-08-07T06:16:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1773 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:56 dist-test-slave-h3ft krb5kdc[8873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:56 dist-test-slave-h3ft krb5kdc[8873](info): set up 2 sockets
Aug 07 06:16:56 dist-test-slave-h3ft krb5kdc[8873](info): commencing operation
krb5kdc: starting...
W20260807 06:16:57.383064 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:57 dist-test-slave-h3ft krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:57.535949 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42197
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:42197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:57.671442 8889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:57.671818 8889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:57.671865 8889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:57.676285 8889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:57.676357 8889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:57.676378 8889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:57.676394 8889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:57.676410 8889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:57.682314 8889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:42197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:42197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:57.683895 8889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:57.685343 8889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:57.695921 8895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:57.695926 8896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:57.695926 8898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:57.696393 8889 server_base.cc:1034] running on GCE node
I20260807 06:16:57.697125 8889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:57.698589 8889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:57.699791 8889 hybrid_clock.cc:648] HybridClock initialized: now 1786083417699739 us; error 65 us; skew 500 ppm
Aug 07 06:16:57 dist-test-slave-h3ft krb5kdc[8873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083417, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:57.703233 8889 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:57.704437 8889 webserver.cc:460] Webserver started at http://127.9.133.190:43509/ using document root <none> and password file <none>
I20260807 06:16:57.705168 8889 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:57.705274 8889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:57.705580 8889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:57.708261 8889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "90c5aa0f3bcb4e3f81000f485f385e65"
format_stamp: "Formatted at 2026-08-07 06:16:57 on dist-test-slave-h3ft"
server_key: "5bdba817dfb0bfaf33a8bb6f481da5c5"
server_key_iv: "173a68881c08d43a26c7fc3d8f71a772"
server_key_version: "encryptionkey@0"
I20260807 06:16:57.709043 8889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "90c5aa0f3bcb4e3f81000f485f385e65"
format_stamp: "Formatted at 2026-08-07 06:16:57 on dist-test-slave-h3ft"
server_key: "5bdba817dfb0bfaf33a8bb6f481da5c5"
server_key_iv: "173a68881c08d43a26c7fc3d8f71a772"
server_key_version: "encryptionkey@0"
I20260807 06:16:57.713910 8889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:16:57.717389 8908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:57.719532 8889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:16:57.719686 8889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "90c5aa0f3bcb4e3f81000f485f385e65"
format_stamp: "Formatted at 2026-08-07 06:16:57 on dist-test-slave-h3ft"
server_key: "5bdba817dfb0bfaf33a8bb6f481da5c5"
server_key_iv: "173a68881c08d43a26c7fc3d8f71a772"
server_key_version: "encryptionkey@0"
I20260807 06:16:57.719892 8889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:57.725395 8889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:57.729321 8889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:57.729552 8889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:57.738023 8889 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:42197
I20260807 06:16:57.738106 8985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:42197 every 8 connection(s)
I20260807 06:16:57.739287 8889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:57.742774 8986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:57.745647 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 8889
I20260807 06:16:57.745821 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:57.746212 9750 external_mini_cluster.cc:1428] Setting key 71f1823df59a95851982914562378fef
I20260807 06:16:57.750694 8986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65: Bootstrap starting.
I20260807 06:16:57.752761 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:57.748242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36433 (local address 127.9.133.190:42197)
0807 06:16:57.748750 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:57.748768 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:57.748812 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:16:57.749595 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:57.749616 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:57.749963 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:57.750367 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:57.751335 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:57.751341 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:57.751363 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:57.751375 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:57.751436 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:57.751461 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:57.751630 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:57.751708 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 06:16:57.751883 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":144,"threads_started":1}
I20260807 06:16:57.753610 8986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:57.754669 8986 log.cc:824] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:57.756234 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:16:57.757195 8986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65: No bootstrap required, opened a new log
I20260807 06:16:57.760561 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:57.758700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48521 (local address 127.9.133.190:42197)
0807 06:16:57.758838 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:57.758843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:57.759065 (+ 222us) server_negotiation.cc:407] Connection header received
0807 06:16:57.759224 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:57.759226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:57.759255 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:57.759325 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:57.760131 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:57.760134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:57.760136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:16:57.760146 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:57.760174 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:16:57.760184 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:57.760260 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:57.760379 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 06:16:57.760412 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:16:57.762491 8986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42197 } }
I20260807 06:16:57.762789 8986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:57.762850 8986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c5aa0f3bcb4e3f81000f485f385e65, State: Initialized, Role: FOLLOWER
I20260807 06:16:57.763478 8986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42197 } }
I20260807 06:16:57.763672 8986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:57.763752 8986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:57.763855 8986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:57.764873 8986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42197 } }
I20260807 06:16:57.765223 8986 leader_election.cc:302] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c5aa0f3bcb4e3f81000f485f385e65; no voters:
I20260807 06:16:57.765569 8986 leader_election.cc:288] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:57.765657 8997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:57.765935 8997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [term 1 LEADER]: Becoming Leader. State: Replica: 90c5aa0f3bcb4e3f81000f485f385e65, State: Running, Role: LEADER
I20260807 06:16:57.766296 8997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42197 } }
I20260807 06:16:57.767128 8986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:57.768404 8999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c5aa0f3bcb4e3f81000f485f385e65. Latest consensus state: current_term: 1 leader_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42197 } } }
I20260807 06:16:57.768378 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c5aa0f3bcb4e3f81000f485f385e65" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 42197 } } }
I20260807 06:16:57.768563 8999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:57.768582 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65 [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:57.769157 9009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:57.773175 9009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:57.780175 9009 catalog_manager.cc:1347] Generated new cluster ID: a1eb09e6dc0a4af29f5522de491c010c
I20260807 06:16:57.780263 9009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:57.815027 9009 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:57.815519 9009 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:57.821940 9009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c5aa0f3bcb4e3f81000f485f385e65: Generated new TSK 0
I20260807 06:16:57.822948 9009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:57.832816 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 8889
2026-08-07T06:16:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1782 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:57 dist-test-slave-h3ft krb5kdc[9026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:57 dist-test-slave-h3ft krb5kdc[9026](info): set up 2 sockets
Aug 07 06:16:57 dist-test-slave-h3ft krb5kdc[9026](info): commencing operation
krb5kdc: starting...
W20260807 06:16:59.109336 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:16:59 dist-test-slave-h3ft krb5kdc[9026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:16:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:16:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:16:59.268971 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39773
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:39773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:16:59.405036 9043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:16:59.405467 9043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:16:59.405557 9043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:16:59.409847 9043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:16:59.409972 9043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:16:59.410013 9043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:16:59.410053 9043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:16:59.410089 9043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:16:59.415928 9043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:39773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:16:59.417382 9043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:16:59.418717 9043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:16:59.428680 9052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:59.428680 9050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:16:59.428704 9055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:16:59.429062 9043 server_base.cc:1034] running on GCE node
I20260807 06:16:59.429754 9043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:16:59.431221 9043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:16:59.432420 9043 hybrid_clock.cc:648] HybridClock initialized: now 1786083419432401 us; error 37 us; skew 500 ppm
Aug 07 06:16:59 dist-test-slave-h3ft krb5kdc[9026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083419, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:16:59.435958 9043 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:16:59.437131 9043 webserver.cc:460] Webserver started at http://127.9.133.190:36543/ using document root <none> and password file <none>
I20260807 06:16:59.437810 9043 fs_manager.cc:359] Metadata directory not provided
I20260807 06:16:59.437876 9043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:16:59.438095 9043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:16:59.440771 9043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "219905646d694f828716dac1e101c08d"
format_stamp: "Formatted at 2026-08-07 06:16:59 on dist-test-slave-h3ft"
server_key: "73485cbce0ee15d6c39c5c19bfe49b27"
server_key_iv: "603062facdd47c727f6ca36a3308ea91"
server_key_version: "encryptionkey@0"
I20260807 06:16:59.441411 9043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "219905646d694f828716dac1e101c08d"
format_stamp: "Formatted at 2026-08-07 06:16:59 on dist-test-slave-h3ft"
server_key: "73485cbce0ee15d6c39c5c19bfe49b27"
server_key_iv: "603062facdd47c727f6ca36a3308ea91"
server_key_version: "encryptionkey@0"
I20260807 06:16:59.446910 9043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:16:59.449936 9067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:16:59.451510 9043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 06:16:59.451668 9043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "219905646d694f828716dac1e101c08d"
format_stamp: "Formatted at 2026-08-07 06:16:59 on dist-test-slave-h3ft"
server_key: "73485cbce0ee15d6c39c5c19bfe49b27"
server_key_iv: "603062facdd47c727f6ca36a3308ea91"
server_key_version: "encryptionkey@0"
I20260807 06:16:59.451784 9043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:16:59.466173 9043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:16:59.470376 9043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:16:59.470630 9043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:16:59.479952 9043 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:39773
I20260807 06:16:59.479971 9133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:39773 every 8 connection(s)
I20260807 06:16:59.481318 9043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:16:59.485045 9134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:16:59.489502 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9043
I20260807 06:16:59.489706 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:16:59.490059 9750 external_mini_cluster.cc:1428] Setting key 59627696cac43ffce9b6763395ceb10d
I20260807 06:16:59.492566 9134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d: Bootstrap starting.
I20260807 06:16:59.495438 9134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d: Neither blocks nor log segments found. Creating new log.
I20260807 06:16:59.496371 9134 log.cc:824] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:16:59.498971 9134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d: No bootstrap required, opened a new log
Aug 07 06:16:59 dist-test-slave-h3ft krb5kdc[9026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:16:59.504365 9134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219905646d694f828716dac1e101c08d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39773 } }
I20260807 06:16:59.504739 9134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:16:59.504828 9134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219905646d694f828716dac1e101c08d, State: Initialized, Role: FOLLOWER
I20260807 06:16:59.505452 9134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219905646d694f828716dac1e101c08d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39773 } }
I20260807 06:16:59.505645 9134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:16:59.505735 9134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:16:59.505847 9134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:16:59.507272 9134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219905646d694f828716dac1e101c08d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39773 } }
I20260807 06:16:59.507704 9134 leader_election.cc:302] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 219905646d694f828716dac1e101c08d; no voters:
I20260807 06:16:59.508076 9134 leader_election.cc:288] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:16:59.508227 9139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:16:59.508498 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:59.491949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54981 (local address 127.9.133.190:39773)
0807 06:16:59.492524 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:59.492543 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:59.492606 (+ 63us) server_negotiation.cc:407] Connection header received
0807 06:16:59.493488 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:59.493511 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:59.493933 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:59.494350 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:59.495488 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:59.496757 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:59.498004 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:59.498579 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:59.498722 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:59.502155 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:59.502164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:59.502190 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:59.502235 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:59.504531 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:59.505164 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:59.505170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:59.505176 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:59.505330 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:59.505640 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:59.505644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:59.505645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:59.506287 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:59.506506 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:59.506839 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 06:16:59.507254 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":172,"threads_started":1}
I20260807 06:16:59.508564 9139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [term 1 LEADER]: Becoming Leader. State: Replica: 219905646d694f828716dac1e101c08d, State: Running, Role: LEADER
I20260807 06:16:59.508962 9139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219905646d694f828716dac1e101c08d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39773 } }
I20260807 06:16:59.509589 9134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:16:59.510346 9139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 219905646d694f828716dac1e101c08d. Latest consensus state: current_term: 1 leader_uuid: "219905646d694f828716dac1e101c08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219905646d694f828716dac1e101c08d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39773 } } }
I20260807 06:16:59.510486 9139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:59.510974 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "219905646d694f828716dac1e101c08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219905646d694f828716dac1e101c08d" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39773 } } }
I20260807 06:16:59.511197 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d [sys.catalog]: This master's current role is: LEADER
I20260807 06:16:59.512274 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 06:16:59.515694 9162 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:16:59.515777 9162 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:16:59.515875 9149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:16:59.516732 9149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:16:59.524235 9149 catalog_manager.cc:1347] Generated new cluster ID: f06d3b641092409ea8710c91a2c5ceae
I20260807 06:16:59.524302 9149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:16:59.524788 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:16:59.515465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45115 (local address 127.9.133.190:39773)
0807 06:16:59.515590 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 06:16:59.515594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:16:59.516140 (+ 546us) server_negotiation.cc:407] Connection header received
0807 06:16:59.516304 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:16:59.516307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:16:59.516359 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:16:59.516459 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:16:59.517269 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:59.517845 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:59.518711 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:16:59.518965 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:16:59.519124 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:16:59.520644 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:16:59.520648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:16:59.520651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:16:59.520683 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:16:59.522411 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:59.522969 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:59.522972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:59.522974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:59.523102 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:16:59.523502 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:16:59.523505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:16:59.523506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:16:59.523746 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:16:59.523887 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 06:16:59.524443 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 06:16:59.524598 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:16:59.532295 9149 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:16:59.533552 9149 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:16:59.543049 9149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 219905646d694f828716dac1e101c08d: Generated new TSK 0
I20260807 06:16:59.544159 9149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:16:59.557235 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9043
2026-08-07T06:16:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1724 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:16:59 dist-test-slave-h3ft krb5kdc[9173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:16:59 dist-test-slave-h3ft krb5kdc[9173](info): set up 2 sockets
Aug 07 06:16:59 dist-test-slave-h3ft krb5kdc[9173](info): commencing operation
krb5kdc: starting...
W20260807 06:17:00.836318 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:17:00 dist-test-slave-h3ft krb5kdc[9173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:17:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:17:00.995739 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40401
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:40401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:17:01.130611 9192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:01.131021 9192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:01.131160 9192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:01.135440 9192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:01.135572 9192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:01.135608 9192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:01.135646 9192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:01.135679 9192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:01.141702 9192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:40401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:40401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:01.143251 9192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:01.144676 9192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:01.155555 9199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:01.155606 9192 server_base.cc:1034] running on GCE node
W20260807 06:17:01.155555 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:01.155555 9202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:01.156582 9192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:01.157979 9192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:01.159138 9192 hybrid_clock.cc:648] HybridClock initialized: now 1786083421159108 us; error 46 us; skew 500 ppm
Aug 07 06:17:01 dist-test-slave-h3ft krb5kdc[9173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083421, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:01.168179 9192 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:17:01.169313 9192 webserver.cc:460] Webserver started at http://127.9.133.190:42451/ using document root <none> and password file <none>
I20260807 06:17:01.170006 9192 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:01.170076 9192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:01.170286 9192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:01.172914 9192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "c1ae17ba8e1f4c31bef45820abb4d58b"
format_stamp: "Formatted at 2026-08-07 06:17:01 on dist-test-slave-h3ft"
server_key: "dbcde9ca4fca0d8c9e3b20a8ee36783f"
server_key_iv: "dae3fdfd20aa6affc7cd7f47f634e2b6"
server_key_version: "encryptionkey@0"
I20260807 06:17:01.173589 9192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "c1ae17ba8e1f4c31bef45820abb4d58b"
format_stamp: "Formatted at 2026-08-07 06:17:01 on dist-test-slave-h3ft"
server_key: "dbcde9ca4fca0d8c9e3b20a8ee36783f"
server_key_iv: "dae3fdfd20aa6affc7cd7f47f634e2b6"
server_key_version: "encryptionkey@0"
I20260807 06:17:01.177977 9192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 06:17:01.180804 9213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:01.182507 9192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:17:01.182662 9192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "c1ae17ba8e1f4c31bef45820abb4d58b"
format_stamp: "Formatted at 2026-08-07 06:17:01 on dist-test-slave-h3ft"
server_key: "dbcde9ca4fca0d8c9e3b20a8ee36783f"
server_key_iv: "dae3fdfd20aa6affc7cd7f47f634e2b6"
server_key_version: "encryptionkey@0"
I20260807 06:17:01.182773 9192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:01.215167 9192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:01.219415 9192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:01.219697 9192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:01.229328 9192 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:40401
I20260807 06:17:01.229344 9282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:40401 every 8 connection(s)
I20260807 06:17:01.230618 9192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:17:01.234491 9283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:01.236428 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9192
I20260807 06:17:01.236622 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:17:01.237056 9750 external_mini_cluster.cc:1428] Setting key f1e7c3e065e027a6b4110a82c41c5215
I20260807 06:17:01.242877 9283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b: Bootstrap starting.
I20260807 06:17:01.246308 9283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:01.247365 9283 log.cc:824] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:01.248997 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:01.238866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49073 (local address 127.9.133.190:40401)
0807 06:17:01.239425 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:01.239443 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:01.239488 (+ 45us) server_negotiation.cc:407] Connection header received
0807 06:17:01.240266 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:01.240288 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:01.240705 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:01.241044 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:01.241583 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:01.242654 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:01.244082 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:01.244698 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:01.244971 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:01.246245 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:01.246251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:01.246270 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:17:01.246871 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:01.246955 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:17:01.246985 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:01.247188 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:01.247356 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 06:17:01.247839 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":225,"threads_started":1}
I20260807 06:17:01.250247 9283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b: No bootstrap required, opened a new log
I20260807 06:17:01.252794 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:17:01.254787 9283 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40401 } }
I20260807 06:17:01.255162 9283 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:17:01.255239 9283 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ae17ba8e1f4c31bef45820abb4d58b, State: Initialized, Role: FOLLOWER
I20260807 06:17:01.255827 9283 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40401 } }
I20260807 06:17:01.256009 9283 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:17:01.256098 9283 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:17:01.256208 9283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:01.257709 9283 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40401 } }
I20260807 06:17:01.258148 9283 leader_election.cc:302] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ae17ba8e1f4c31bef45820abb4d58b; no voters:
I20260807 06:17:01.258553 9283 leader_election.cc:288] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:17:01.258633 9300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:17:01.258864 9300 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [term 1 LEADER]: Becoming Leader. State: Replica: c1ae17ba8e1f4c31bef45820abb4d58b, State: Running, Role: LEADER
I20260807 06:17:01.259238 9300 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40401 } }
I20260807 06:17:01.260232 9283 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:01.260721 9302 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40401 } } }
I20260807 06:17:01.260896 9302 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:01.261091 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:01.255619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53413 (local address 127.9.133.190:40401)
0807 06:17:01.255767 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:01.255772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:01.256064 (+ 292us) server_negotiation.cc:407] Connection header received
0807 06:17:01.256281 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:01.256285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:01.256347 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:01.256454 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:01.256750 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:01.257559 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:01.258678 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:01.259075 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:01.259207 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:01.260224 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:01.260229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:01.260232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:17:01.260246 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:01.260283 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:17:01.260299 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:01.260377 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:01.260806 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 06:17:01.260883 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 06:17:01.261214 9301 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ae17ba8e1f4c31bef45820abb4d58b. Latest consensus state: current_term: 1 leader_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ae17ba8e1f4c31bef45820abb4d58b" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 40401 } } }
I20260807 06:17:01.261332 9301 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:01.261703 9304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:17:01.265864 9304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:17:01.273793 9304 catalog_manager.cc:1347] Generated new cluster ID: 063005a14eab4f289cbb6d4df600dc7e
I20260807 06:17:01.273917 9304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:17:01.294319 9304 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:17:01.295548 9304 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:17:01.304395 9304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1ae17ba8e1f4c31bef45820abb4d58b: Generated new TSK 0
I20260807 06:17:01.305332 9304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:17:01.316692 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9192
2026-08-07T06:17:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1758 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:17:01 dist-test-slave-h3ft krb5kdc[9334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:17:01 dist-test-slave-h3ft krb5kdc[9334](info): set up 2 sockets
Aug 07 06:17:01 dist-test-slave-h3ft krb5kdc[9334](info): commencing operation
krb5kdc: starting...
W20260807 06:17:02.594787 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:17:02 dist-test-slave-h3ft krb5kdc[9334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:17:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:17:02.759624 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:37841
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:17:02.903604 9352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:02.903987 9352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:02.904038 9352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:02.908752 9352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:02.908834 9352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:02.908865 9352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:02.908883 9352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:02.908900 9352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:02.914850 9352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:37841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:02.916340 9352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:02.917660 9352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:02.928508 9359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:02.928524 9363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:02.928503 9360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:02.928695 9352 server_base.cc:1034] running on GCE node
I20260807 06:17:02.929517 9352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:02.930962 9352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:02.932220 9352 hybrid_clock.cc:648] HybridClock initialized: now 1786083422932194 us; error 53 us; skew 500 ppm
Aug 07 06:17:02 dist-test-slave-h3ft krb5kdc[9334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083422, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:02.935953 9352 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:17:02.937127 9352 webserver.cc:460] Webserver started at http://127.9.133.190:39229/ using document root <none> and password file <none>
I20260807 06:17:02.937804 9352 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:02.937879 9352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:02.938098 9352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:02.940826 9352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "465de947ed524be480b21e15f0c4e284"
format_stamp: "Formatted at 2026-08-07 06:17:02 on dist-test-slave-h3ft"
server_key: "8b59ad61e354d052eaf59fb9184e9339"
server_key_iv: "aea2c2b79711eb1bb1539ae8b6f1e738"
server_key_version: "encryptionkey@0"
I20260807 06:17:02.941455 9352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "465de947ed524be480b21e15f0c4e284"
format_stamp: "Formatted at 2026-08-07 06:17:02 on dist-test-slave-h3ft"
server_key: "8b59ad61e354d052eaf59fb9184e9339"
server_key_iv: "aea2c2b79711eb1bb1539ae8b6f1e738"
server_key_version: "encryptionkey@0"
I20260807 06:17:02.946309 9352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 06:17:02.949604 9370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:02.951253 9352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:17:02.951433 9352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "465de947ed524be480b21e15f0c4e284"
format_stamp: "Formatted at 2026-08-07 06:17:02 on dist-test-slave-h3ft"
server_key: "8b59ad61e354d052eaf59fb9184e9339"
server_key_iv: "aea2c2b79711eb1bb1539ae8b6f1e738"
server_key_version: "encryptionkey@0"
I20260807 06:17:02.951576 9352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:02.957391 9352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:02.961732 9352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:02.961993 9352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:02.970983 9352 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:37841
I20260807 06:17:02.971030 9443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:37841 every 8 connection(s)
I20260807 06:17:02.972231 9352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:17:02.975795 9444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:02.979650 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9352
I20260807 06:17:02.979815 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:17:02.980149 9750 external_mini_cluster.cc:1428] Setting key a173874bc97efa78c0dfb5933264b913
I20260807 06:17:02.983614 9444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284: Bootstrap starting.
I20260807 06:17:02.987068 9444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:02.987136 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:02.981946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60237 (local address 127.9.133.190:37841)
0807 06:17:02.982742 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:02.982762 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:02.982811 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:17:02.983740 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:02.983762 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:02.984176 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:02.984586 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:02.985643 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:02.985649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:02.985662 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:17:02.985674 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:02.985741 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:17:02.985763 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:02.985926 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:02.986002 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 06:17:02.986221 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":176,"threads_started":1}
I20260807 06:17:02.988306 9444 log.cc:824] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:02.991138 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:17:02.991369 9444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284: No bootstrap required, opened a new log
I20260807 06:17:02.996106 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:02.993825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39075 (local address 127.9.133.190:37841)
0807 06:17:02.993989 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:02.993995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:02.994262 (+ 267us) server_negotiation.cc:407] Connection header received
0807 06:17:02.994414 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:02.994417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:02.994456 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:02.994529 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:02.995576 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:02.995580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:02.995582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 06:17:02.995593 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:02.995624 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 06:17:02.995636 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:02.995717 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:02.995856 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 06:17:02.995895 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:17:02.997112 9444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465de947ed524be480b21e15f0c4e284" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37841 } }
I20260807 06:17:02.997458 9444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 06:17:02.997550 9444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 465de947ed524be480b21e15f0c4e284, State: Initialized, Role: FOLLOWER
I20260807 06:17:02.998181 9444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465de947ed524be480b21e15f0c4e284" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37841 } }
I20260807 06:17:02.998369 9444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 06:17:02.998458 9444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 06:17:02.998590 9444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:02.999958 9444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465de947ed524be480b21e15f0c4e284" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37841 } }
I20260807 06:17:03.000439 9444 leader_election.cc:302] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 465de947ed524be480b21e15f0c4e284; no voters:
I20260807 06:17:03.000890 9444 leader_election.cc:288] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 06:17:03.001087 9457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:17:03.001415 9457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [term 1 LEADER]: Becoming Leader. State: Replica: 465de947ed524be480b21e15f0c4e284, State: Running, Role: LEADER
I20260807 06:17:03.001832 9457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465de947ed524be480b21e15f0c4e284" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37841 } }
I20260807 06:17:03.002827 9444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:03.003942 9461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 465de947ed524be480b21e15f0c4e284. Latest consensus state: current_term: 1 leader_uuid: "465de947ed524be480b21e15f0c4e284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465de947ed524be480b21e15f0c4e284" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37841 } } }
I20260807 06:17:03.004112 9461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:03.004557 9459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "465de947ed524be480b21e15f0c4e284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465de947ed524be480b21e15f0c4e284" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 37841 } } }
I20260807 06:17:03.004727 9459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:03.005120 9468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:17:03.009375 9468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:17:03.016741 9468 catalog_manager.cc:1347] Generated new cluster ID: e89b0cccbd0a44cbb9ed0a811ea0bda3
I20260807 06:17:03.016866 9468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:17:03.026266 9468 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:17:03.026697 9468 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:17:03.034898 9468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 465de947ed524be480b21e15f0c4e284: Generated new TSK 0
I20260807 06:17:03.035935 9468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:17:03.045125 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9352
2026-08-07T06:17:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1730 ms)
[----------] 18 tests from AuthTokenIssuingTest (31626 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:17:03 dist-test-slave-h3ft krb5kdc[9494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:17:03 dist-test-slave-h3ft krb5kdc[9494](info): set up 2 sockets
Aug 07 06:17:03 dist-test-slave-h3ft krb5kdc[9494](info): commencing operation
krb5kdc: starting...
W20260807 06:17:04.329707 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:17:04 dist-test-slave-h3ft krb5kdc[9494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:17:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:17:04.487588 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39065
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:39065,127.9.133.189:34677,127.9.133.188:40355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:17:04.623907 9511 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:17:04.624351 9511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:04.624430 9511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:04.624524 9511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:04.628852 9511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:04.628957 9511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:04.628993 9511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:04.629036 9511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:04.629096 9511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:04.634922 9511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:39065,127.9.133.189:34677,127.9.133.188:40355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:39065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:04.636443 9511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:04.637804 9511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:04.648620 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:04.648620 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:04.648604 9523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:04.648718 9511 server_base.cc:1034] running on GCE node
I20260807 06:17:04.649581 9511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:04.651116 9511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:04.652295 9511 hybrid_clock.cc:648] HybridClock initialized: now 1786083424652267 us; error 44 us; skew 500 ppm
Aug 07 06:17:04 dist-test-slave-h3ft krb5kdc[9494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:04.655686 9511 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:17:04.656829 9511 webserver.cc:460] Webserver started at http://127.9.133.190:37815/ using document root <none> and password file <none>
I20260807 06:17:04.657550 9511 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:04.657620 9511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:04.657881 9511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:04.660555 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "8dc91e2fbabb4e7886918c61ddbb7276"
format_stamp: "Formatted at 2026-08-07 06:17:04 on dist-test-slave-h3ft"
server_key: "9e6cf327985c041b6dcb730d5fd2361a"
server_key_iv: "b32726a8484853af42c0c18ae6d2d6fd"
server_key_version: "encryptionkey@0"
I20260807 06:17:04.661221 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "8dc91e2fbabb4e7886918c61ddbb7276"
format_stamp: "Formatted at 2026-08-07 06:17:04 on dist-test-slave-h3ft"
server_key: "9e6cf327985c041b6dcb730d5fd2361a"
server_key_iv: "b32726a8484853af42c0c18ae6d2d6fd"
server_key_version: "encryptionkey@0"
I20260807 06:17:04.665745 9511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 06:17:04.668808 9531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:04.670459 9511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 06:17:04.670610 9511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "8dc91e2fbabb4e7886918c61ddbb7276"
format_stamp: "Formatted at 2026-08-07 06:17:04 on dist-test-slave-h3ft"
server_key: "9e6cf327985c041b6dcb730d5fd2361a"
server_key_iv: "b32726a8484853af42c0c18ae6d2d6fd"
server_key_version: "encryptionkey@0"
I20260807 06:17:04.670737 9511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:04.687923 9511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:04.691896 9511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:04.692164 9511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:04.701259 9511 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:39065
I20260807 06:17:04.701646 9605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:39065 every 8 connection(s)
I20260807 06:17:04.702522 9511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:17:04.706288 9606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:04.707914 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9511
I20260807 06:17:04.708088 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:17:04.708431 9750 external_mini_cluster.cc:1428] Setting key b446d90db2762e3147e1592775f81c30
I20260807 06:17:04.711496 9606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.133.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.189@KRBTEST.COM" created.
Aug 07 06:17:04 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:17:04.729928 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.714568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.9.133.190:39065)
0807 06:17:04.715164 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:04.715187 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:04.715519 (+ 332us) server_negotiation.cc:407] Connection header received
0807 06:17:04.716794 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:04.716798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:04.717015 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:04.717240 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:04.718221 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:04.719322 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:04.720420 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:04.720824 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:04.721026 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:04.723934 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:04.723942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:04.723953 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:04.723976 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:04.726188 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:04.726726 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:04.726732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:04.726733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:04.726883 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:04.727281 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:04.727286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:04.727288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:04.727763 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:04.727917 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:04.728323 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 06:17:04.728615 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":185,"threads_started":1}
I20260807 06:17:04.729928 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.714569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:39065 (local address 127.0.0.1:41684)
0807 06:17:04.715334 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.715378 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:04.716430 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:04.717241 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:04.717246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:04.717746 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:04.718053 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:04.718069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:04.719475 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:04.719479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:04.720282 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:04.720292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:04.721011 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:04.721015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:04.721132 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:04.721870 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:04.721897 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:04.723761 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:04.726320 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:04.726330 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:04.726341 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:04.726614 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:04.727041 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:04.727044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:04.727048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:04.727177 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:04.727892 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:04.727900 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:04.728056 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 06:17:04.728245 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 06:17:04.728707 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":115,"spinlock_wait_cycles":40064,"thread_start_us":186,"threads_started":1}
I20260807 06:17:04.733139 9606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:04.734087 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.733774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53212)
0807 06:17:04.733920 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.734003 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 06:17:04.734339 9536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
I20260807 06:17:04.735494 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.735244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53224)
0807 06:17:04.735371 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.735426 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 06:17:04.735693 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34677: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab.
Entry for principal kudu/127.9.133.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab.
I20260807 06:17:04.743790 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:34677
--webserver_interface=127.9.133.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab
--principal=kudu/127.9.133.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:39065,127.9.133.189:34677,127.9.133.188:40355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:17:04.785015 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } attempt: 1
I20260807 06:17:04.786264 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.785867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53230)
0807 06:17:04.786071 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.786151 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 06:17:04.787261 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.786920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53232)
0807 06:17:04.787120 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.787179 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 06:17:04.787423 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34677: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
I20260807 06:17:04.855605 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } attempt: 2
I20260807 06:17:04.856827 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.856420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53238)
0807 06:17:04.856618 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.856704 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 06:17:04.857746 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.857470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53244)
0807 06:17:04.857607 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.857666 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 06:17:04.857924 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34677: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
W20260807 06:17:04.890027 9617 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:17:04.890492 9617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:04.890574 9617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:04.890661 9617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:04.895020 9617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:04.895133 9617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:04.895185 9617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:04.895246 9617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:04.895304 9617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:04.901263 9617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:39065,127.9.133.189:34677,127.9.133.188:40355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab
--principal=kudu/127.9.133.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:34677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.133.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:04.902817 9617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:04.904184 9617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:04.914989 9627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:04.914960 9617 server_base.cc:1034] running on GCE node
W20260807 06:17:04.914981 9623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:04.914942 9622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:04.916249 9617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:04.917827 9617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:04.919111 9617 hybrid_clock.cc:648] HybridClock initialized: now 1786083424919084 us; error 38 us; skew 500 ppm
Aug 07 06:17:04 dist-test-slave-h3ft krb5kdc[9494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:04.922832 9617 init.cc:377] Logged in from keytab as kudu/127.9.133.189@KRBTEST.COM (short username kudu)
I20260807 06:17:04.924378 9617 webserver.cc:460] Webserver started at http://127.9.133.189:42193/ using document root <none> and password file <none>
I20260807 06:17:04.925177 9617 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:04.925287 9617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:04.925599 9617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:04.928495 9617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data/instance:
uuid: "2dcf06b5cbe048c28f50e7660a0441dc"
format_stamp: "Formatted at 2026-08-07 06:17:04 on dist-test-slave-h3ft"
server_key: "8dc2e604c8e03cb82d8a58187171ceed"
server_key_iv: "cb46ab4dce887b769a2a7a098a058f92"
server_key_version: "encryptionkey@0"
I20260807 06:17:04.929263 9617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal/instance:
uuid: "2dcf06b5cbe048c28f50e7660a0441dc"
format_stamp: "Formatted at 2026-08-07 06:17:04 on dist-test-slave-h3ft"
server_key: "8dc2e604c8e03cb82d8a58187171ceed"
server_key_iv: "cb46ab4dce887b769a2a7a098a058f92"
server_key_version: "encryptionkey@0"
I20260807 06:17:04.934725 9617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 06:17:04.938566 9638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:04.940500 9617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:17:04.940735 9617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal
uuid: "2dcf06b5cbe048c28f50e7660a0441dc"
format_stamp: "Formatted at 2026-08-07 06:17:04 on dist-test-slave-h3ft"
server_key: "8dc2e604c8e03cb82d8a58187171ceed"
server_key_iv: "cb46ab4dce887b769a2a7a098a058f92"
server_key_version: "encryptionkey@0"
I20260807 06:17:04.940923 9617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:04.949155 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } attempt: 3
I20260807 06:17:04.950438 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.950008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53248)
0807 06:17:04.950234 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.950327 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 06:17:04.951448 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.951178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53264)
0807 06:17:04.951309 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.951373 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 06:17:04.951632 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:34677: Network error: Client connection negotiation failed: client connection to 127.9.133.189:34677: connect: Connection refused (error 111)
I20260807 06:17:04.970320 9617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:04.974917 9617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:04.975240 9617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:04.984499 9617 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.189:34677
I20260807 06:17:04.984864 9709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.189:34677 every 8 connection(s)
I20260807 06:17:04.985947 9617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
I20260807 06:17:04.989934 9710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:04.995163 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9617
I20260807 06:17:04.995319 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-1/wal/instance
I20260807 06:17:04.995693 9750 external_mini_cluster.cc:1428] Setting key a7e8cc2ee2ca169207a072325b5be4c7
I20260807 06:17:04.995555 9710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.133.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.188@KRBTEST.COM" created.
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:17:05.020382 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.998360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.9.133.190:39065)
0807 06:17:04.998619 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:04.998626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:04.999806 (+ 1180us) server_negotiation.cc:407] Connection header received
0807 06:17:05.000983 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.000990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.001087 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.001219 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.002876 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.004014 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.005621 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.006112 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.006264 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.013334 (+ 7070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.013339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.013343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.013389 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.016824 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.017719 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.017727 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.017733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.017878 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.018369 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.018376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.018382 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.018782 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.018939 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.019851 (+ 912us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.020072 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260807 06:17:05.021399 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:04.998875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:39065 (local address 127.0.0.1:41690)
0807 06:17:04.999580 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:04.999644 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.000708 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.001425 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.001433 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.001900 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.002647 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.002667 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.004233 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.004237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.005456 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.005469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.006365 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.006369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.006599 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.007526 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.007547 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.012928 (+ 5381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.017057 (+ 4129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.017066 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.017093 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.017567 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.018024 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.018027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.018029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.018244 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.018924 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.018933 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.019504 (+ 571us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.019725 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.020158 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":183,"threads_started":1}
I20260807 06:17:05.023540 9710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab.
Entry for principal kudu/127.9.133.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab.
I20260807 06:17:05.035552 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:40355
--webserver_interface=127.9.133.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab
--principal=kudu/127.9.133.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:39065,127.9.133.189:34677,127.9.133.188:40355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:17:05.036026 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.024357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53276)
0807 06:17:05.024491 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.024508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.024593 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.025297 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.025300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.025515 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.025651 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.025658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.026983 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.026986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.027856 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.027866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.028767 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.028770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.028851 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.029528 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.029551 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.031313 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.033908 (+ 2595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.033911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.033914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.034220 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.034582 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.034585 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.034588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.034680 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.035349 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.035352 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.035478 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.035603 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.035785 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 06:17:05.036094 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.024481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.9.133.189:34677)
0807 06:17:05.024871 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.024882 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.024903 (+ 21us) server_negotiation.cc:407] Connection header received
0807 06:17:05.024939 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.024944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.025129 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.025252 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.025784 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.026834 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.028011 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.028644 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.028748 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.031487 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.031504 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.031520 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.031563 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.033766 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.034375 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.034381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.034383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.034472 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.034782 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.034788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.034790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.035222 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.035393 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.035712 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.035879 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":178,"threads_started":1}
I20260807 06:17:05.038568 9710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:05.039624 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.039244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46096)
0807 06:17:05.039407 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.039517 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 06:17:05.039893 9640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.040984 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.040683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46110)
0807 06:17:05.040832 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.040896 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 06:17:05.041180 9710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.090552 9710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 1
I20260807 06:17:05.091952 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.091499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46116)
0807 06:17:05.091726 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.091820 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 06:17:05.092957 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.092685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46126)
0807 06:17:05.092833 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.092889 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 06:17:05.093132 9710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.094893 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } attempt: 4
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
I20260807 06:17:05.107048 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.095783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.9.133.189:34677)
0807 06:17:05.095910 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.095915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.096039 (+ 124us) server_negotiation.cc:407] Connection header received
0807 06:17:05.096261 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.096264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.096316 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.096416 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.097045 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.097667 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.098559 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.098848 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.098968 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.102396 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.102401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.102404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.102443 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.104742 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.105370 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.105375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.105377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.105494 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.105865 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.105869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.105870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.106162 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.106273 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.106659 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.106817 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 06:17:05.107085 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.095668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53280)
0807 06:17:05.095925 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.095945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.096119 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.096446 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.096449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.096720 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.096855 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.096862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.097793 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.097797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.098421 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.098432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.098990 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.098993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.099164 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.099975 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.100011 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.102118 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.104947 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.104951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.104953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.105236 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.105639 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.105642 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.105643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.105756 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.106308 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.106312 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.106454 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.106587 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.106774 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 06:17:05.108831 9606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:05.109877 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.109535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46132)
0807 06:17:05.109711 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.109779 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 06:17:05.110795 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.110558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46142)
0807 06:17:05.110675 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.110731 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 06:17:05.111024 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.161381 9710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 2
I20260807 06:17:05.162657 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.162196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46148)
0807 06:17:05.162423 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.162517 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 06:17:05.163800 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.163486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46152)
0807 06:17:05.163641 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.163712 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 06:17:05.164016 9710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.170243 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 1
I20260807 06:17:05.171278 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.170870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46160)
0807 06:17:05.171083 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.171167 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 06:17:05.172302 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.172025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46166)
0807 06:17:05.172163 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.172227 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 06:17:05.172526 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
W20260807 06:17:05.182960 9716 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:17:05.183511 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:05.183601 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:05.183689 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:05.188053 9716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:05.188122 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:05.188143 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:05.188159 9716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:05.188175 9716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:05.194383 9716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:39065,127.9.133.189:34677,127.9.133.188:40355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab
--principal=kudu/127.9.133.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:40355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.133.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:05.195884 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:05.197368 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:05.208365 9729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:05.208424 9731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:05.208825 9716 server_base.cc:1034] running on GCE node
W20260807 06:17:05.208365 9728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:05.209720 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:05.211337 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:05.212571 9716 hybrid_clock.cc:648] HybridClock initialized: now 1786083425212550 us; error 37 us; skew 500 ppm
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083425, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:05.216359 9716 init.cc:377] Logged in from keytab as kudu/127.9.133.188@KRBTEST.COM (short username kudu)
I20260807 06:17:05.217535 9716 webserver.cc:460] Webserver started at http://127.9.133.188:35415/ using document root <none> and password file <none>
I20260807 06:17:05.218303 9716 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:05.218426 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:05.218730 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:05.221480 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data/instance:
uuid: "60db51a36da54f798f6e083a551857d3"
format_stamp: "Formatted at 2026-08-07 06:17:05 on dist-test-slave-h3ft"
server_key: "e18cc1a0eec4aa20fded51135eeb4edf"
server_key_iv: "ebbd7483873ccf0aca2daec7380dc523"
server_key_version: "encryptionkey@0"
I20260807 06:17:05.222184 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal/instance:
uuid: "60db51a36da54f798f6e083a551857d3"
format_stamp: "Formatted at 2026-08-07 06:17:05 on dist-test-slave-h3ft"
server_key: "e18cc1a0eec4aa20fded51135eeb4edf"
server_key_iv: "ebbd7483873ccf0aca2daec7380dc523"
server_key_version: "encryptionkey@0"
I20260807 06:17:05.226991 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 06:17:05.230079 9742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:05.231739 9716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:17:05.231906 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal
uuid: "60db51a36da54f798f6e083a551857d3"
format_stamp: "Formatted at 2026-08-07 06:17:05 on dist-test-slave-h3ft"
server_key: "e18cc1a0eec4aa20fded51135eeb4edf"
server_key_iv: "ebbd7483873ccf0aca2daec7380dc523"
server_key_version: "encryptionkey@0"
I20260807 06:17:05.232062 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:05.239722 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 2
I20260807 06:17:05.240803 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.240438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46168)
0807 06:17:05.240628 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.240701 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 06:17:05.241660 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.241397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46184)
0807 06:17:05.241529 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.241588 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 06:17:05.241823 9606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.252388 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:05.255209 9710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 3
I20260807 06:17:05.256258 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.255852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46188)
0807 06:17:05.256065 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.256155 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 06:17:05.256881 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:05.257133 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:05.257299 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.256893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46194)
0807 06:17:05.257144 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.257211 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260807 06:17:05.257495 9710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:40355: Network error: Client connection negotiation failed: client connection to 127.9.133.188:40355: connect: Connection refused (error 111)
I20260807 06:17:05.266441 9716 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.188:40355
I20260807 06:17:05.266732 9816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.188:40355 every 8 connection(s)
I20260807 06:17:05.267771 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
I20260807 06:17:05.271416 9817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:05.276244 9817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:05.276818 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 9716
I20260807 06:17:05.276993 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786083100784614-9750-0/minicluster-data/master-2/wal/instance
I20260807 06:17:05.277354 9750 external_mini_cluster.cc:1428] Setting key cba6eb8ac4ee800ad7c77b3974c164f5
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083425, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:17:05.293524 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.278750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.9.133.190:39065)
0807 06:17:05.278929 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.278935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.280079 (+ 1144us) server_negotiation.cc:407] Connection header received
0807 06:17:05.281371 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.281374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.281454 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.281568 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.282620 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.283311 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.284507 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.284934 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.285029 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.288464 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.288468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.288470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.288503 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.290627 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.291422 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.291425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.291426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.291527 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.291936 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.291940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.291942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.292282 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.292402 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.293114 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.293282 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:17:05.294590 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.279297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:39065 (local address 127.0.0.1:41692)
0807 06:17:05.279908 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.279958 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.281104 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.281619 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.281624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.282133 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.282423 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.282438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.283430 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.283434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.284379 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.284388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.285104 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.285108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.285441 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.286269 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.286287 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.288261 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.290762 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.290771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.290782 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.291307 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.291636 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.291639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.291642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.291812 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.292419 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.292426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.292725 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.292984 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.293404 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":40,"spinlock_wait_cycles":4736,"thread_start_us":141,"threads_started":1}
I20260807 06:17:05.294595 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.280321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46196 (local address 127.9.133.188:40355)
0807 06:17:05.280654 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.280666 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.280715 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:17:05.281104 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.281130 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.281411 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.281626 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.282587 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.283738 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.284900 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.285551 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.285687 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.288737 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.288747 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.288765 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.288800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.291101 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.291687 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.291693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.291694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.291828 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.292168 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.292171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.292173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.292718 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.292917 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.293231 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.293469 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":226,"spinlock_wait_cycles":8832,"thread_start_us":110,"threads_started":1}
I20260807 06:17:05.297160 9817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083425, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:17:05.307309 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.297985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.9.133.189:34677)
0807 06:17:05.298171 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.298176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.298198 (+ 22us) server_negotiation.cc:407] Connection header received
0807 06:17:05.298243 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.298246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.298290 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.298391 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.298848 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.299492 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.300277 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.300513 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.300622 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.303206 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.303210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.303212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.303242 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.305172 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.305726 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.305732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.305734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.305868 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.306174 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.306178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.306180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.306481 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.306628 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.306930 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.307139 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 06:17:05.307309 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.297805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:34677 (local address 127.0.0.1:53288)
0807 06:17:05.297947 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.297961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.298057 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.298407 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.298409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.298611 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.298719 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.298725 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.299602 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.299607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.300148 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.300156 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.300631 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.300635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.300753 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.301375 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.301389 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.303065 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.305304 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.305311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.305316 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.305612 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.305983 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.305986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.305987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.306072 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.306612 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.306616 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.306741 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.306866 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.307089 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 06:17:05.308378 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.297781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.9.133.190:39065)
0807 06:17:05.298182 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.298189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.298204 (+ 15us) server_negotiation.cc:407] Connection header received
0807 06:17:05.298240 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.298243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.298300 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.298396 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.299185 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.299748 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.300622 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.300876 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.301025 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.304316 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.304323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.304327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.304360 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.306363 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.306820 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.306826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.306828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.306981 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.307367 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.307371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.307373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.307651 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.307976 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.308061 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.308197 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260807 06:17:05.308614 9817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083425, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
I20260807 06:17:05.319399 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.309245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46200)
0807 06:17:05.309593 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.309607 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.309726 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.310022 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.310025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.310243 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.310365 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.310372 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.311585 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.311589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.312320 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.312329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.313064 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.313068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.313156 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.313748 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.313761 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.315180 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.317375 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.317381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.317385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.317677 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.318136 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.318140 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.318143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.318260 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.318704 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.318709 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.318833 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.318964 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.319167 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260807 06:17:05.319437 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.309312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.9.133.188:40355)
0807 06:17:05.309441 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.309445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.309679 (+ 234us) server_negotiation.cc:407] Connection header received
0807 06:17:05.309839 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.309843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.309907 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.309992 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.310492 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.311462 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.312455 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.312934 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.313047 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.315345 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.315353 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.315358 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.315392 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.317228 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.317841 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.317846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.317849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.318018 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.318392 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.318395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.318396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.318589 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.318675 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.319100 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.319228 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 06:17:05.320825 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.309808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.9.133.189:34677)
0807 06:17:05.309931 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.309935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.310070 (+ 135us) server_negotiation.cc:407] Connection header received
0807 06:17:05.310192 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.310195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.310246 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.310333 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.311199 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.311927 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.313048 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.313419 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.313506 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.316760 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.316764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.316767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.316803 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.318878 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.319433 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.319439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.319444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.319556 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.319948 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.319951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.319952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.320135 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.320333 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.320521 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.320619 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 06:17:05.324967 9817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: Bootstrap starting.
I20260807 06:17:05.325198 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:17:05.326894 9566 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8dc91e2fbabb4e7886918c61ddbb7276"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41698
I20260807 06:17:05.328744 9817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:05.329917 9817 log.cc:824] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:05.332942 9817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: No bootstrap required, opened a new log
I20260807 06:17:05.338218 9817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.338563 9817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db51a36da54f798f6e083a551857d3, State: Initialized, Role: FOLLOWER
I20260807 06:17:05.339082 9817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.339731 9825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.339912 9825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:05.340987 9817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:05.341984 9606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 3
W20260807 06:17:05.347846 9842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:17:05.348002 9842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:17:05.346545 9716 master_runner.cc:186] Error getting master registration for 127.9.133.190:39065: OK, instance_id { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" instance_seqno: 1786083424698691 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:17:05.350288 9716 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:17:05.355050 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.342752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46212)
0807 06:17:05.342973 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.342989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.343196 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.343601 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.343604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.343904 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.344087 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.344096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.345224 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.345228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.346050 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.346063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.348161 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.348165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.348282 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.348961 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.348978 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.350877 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.353116 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.353122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.353127 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.353422 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.353740 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.353743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.353746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.353862 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.354379 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.354383 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.354510 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.354634 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.354809 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 06:17:05.355681 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.342769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.9.133.188:40355)
0807 06:17:05.342903 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.342908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.343116 (+ 208us) server_negotiation.cc:407] Connection header received
0807 06:17:05.343367 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.343372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.343443 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.343538 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.344294 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.345068 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.346182 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.346504 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.346619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.351060 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.351064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.351067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.351101 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.352983 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.353538 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.353542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.353544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.353639 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.353968 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.353972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.353974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.354261 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.354364 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.355370 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.355513 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 06:17:05.359795 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276: Bootstrap starting.
I20260807 06:17:05.363562 9606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:05.364625 9606 log.cc:824] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:05.367125 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276: No bootstrap required, opened a new log
I20260807 06:17:05.369925 9606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.370204 9606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc91e2fbabb4e7886918c61ddbb7276, State: Initialized, Role: FOLLOWER
I20260807 06:17:05.370582 9606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.371234 9847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.371388 9847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:05.372282 9606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:05.372398 9566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 06:17:05.372550 9566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:05.374228 9566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.377339 9781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "60db51a36da54f798f6e083a551857d3"
I20260807 06:17:05.377683 9781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:05.379398 9781 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc91e2fbabb4e7886918c61ddbb7276 in term 1.
W20260807 06:17:05.379877 9863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 06:17:05.379936 9533 leader_election.cc:302] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db51a36da54f798f6e083a551857d3, 8dc91e2fbabb4e7886918c61ddbb7276; no voters:
I20260807 06:17:05.380282 9847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:17:05.380524 9847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [term 1 LEADER]: Becoming Leader. State: Replica: 8dc91e2fbabb4e7886918c61ddbb7276, State: Running, Role: LEADER
I20260807 06:17:05.380918 9847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.378470 9511 master_runner.cc:186] Error getting master registration for 127.9.133.189:34677: OK, instance_id { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" instance_seqno: 1786083424981975 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:17:05.382166 9511 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:17:05.383098 9566 leader_election.cc:288] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [CANDIDATE]: Term 1 election: Requested vote from peers 2dcf06b5cbe048c28f50e7660a0441dc (127.9.133.189:34677), 60db51a36da54f798f6e083a551857d3 (127.9.133.188:40355)
W20260807 06:17:05.380030 9863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:17:05.385102 9866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dc91e2fbabb4e7886918c61ddbb7276. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.385263 9866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:05.385910 9873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:17:05.387858 9873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:17:05.396747 9781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [term 1 FOLLOWER]: Refusing update from remote peer 8dc91e2fbabb4e7886918c61ddbb7276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:17:05.397850 9866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:17:05.400655 9710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } attempt: 4
I20260807 06:17:05.404114 9825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dc91e2fbabb4e7886918c61ddbb7276. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.404282 9825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:05.408991 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.391530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.9.133.190:39065)
0807 06:17:05.391706 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.391712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.393202 (+ 1490us) server_negotiation.cc:407] Connection header received
0807 06:17:05.393558 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.393562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.393627 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.393686 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.395644 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.396522 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.398581 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.398940 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.399047 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.403105 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.403110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.403113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.403151 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.405470 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.406483 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.406487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.406490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.406615 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.407274 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.407281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.407285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.407626 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.407763 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.408599 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.408763 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 06:17:05.411868 9847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.412110 9847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:05.412191 9825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.412267 9874 mvcc.cc:205] Tried to move back new op lower bound from 7315797710413905920 to 7315797710364123136. Current Snapshot: MvccSnapshot[applied={T|T < 7315797710413905920}]
I20260807 06:17:05.412335 9825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:05.412983 9880 mvcc.cc:205] Tried to move back new op lower bound from 7315797710413905920 to 7315797710364123136. Current Snapshot: MvccSnapshot[applied={T|T < 7315797710413905920}]
I20260807 06:17:05.419880 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.392411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.9.133.189:34677)
0807 06:17:05.392569 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.392575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.399924 (+ 7349us) server_negotiation.cc:407] Connection header received
0807 06:17:05.400156 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.400164 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.400250 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.400319 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.403089 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.404030 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.406192 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.406605 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.406707 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.411184 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.411189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.411192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.411233 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.413486 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.417970 (+ 4484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.417976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.417980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.418121 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.418509 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.418513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.418515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.418798 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.418962 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.419422 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.419614 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 06:17:05 dist-test-slave-h3ft krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083424, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:17:05.422683 9873 catalog_manager.cc:1347] Generated new cluster ID: e6aae1d4ef2a45a19180318a8d52af37
I20260807 06:17:05.422770 9873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:17:05.427030 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.399422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.9.133.188:40355)
0807 06:17:05.399555 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.399560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.410081 (+ 10521us) server_negotiation.cc:407] Connection header received
0807 06:17:05.410520 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.410524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.410595 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.410657 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.414433 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.415320 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.416654 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.417019 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.417096 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.423095 (+ 5999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.423100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.423103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.423133 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.425058 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.425578 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.425582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.425584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.425669 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.425951 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.425954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.425956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.426174 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.426295 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.426678 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.426810 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:17:05.443032 9873 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:17:05.444635 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.401158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:40355 (local address 127.0.0.1:46228)
0807 06:17:05.401425 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:05.401442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:05.401540 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:05.414657 (+ 13117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:05.414672 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:05.414885 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:05.415044 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.415052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.416945 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.416949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.417894 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:05.417914 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.419108 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.419123 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:05.419222 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.419831 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:05.419846 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:05.421693 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:05.441557 (+ 19864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.441577 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.441580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.441944 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.442740 (+ 796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:05.442747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:05.442749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:05.442876 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:05.443978 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:05.443981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:05.444124 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 06:17:05.444230 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 06:17:05.444426 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260807 06:17:05.444779 9873 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:17:05.448922 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:05.401767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.9.133.188:40355)
0807 06:17:05.414142 (+ 12375us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.414180 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.414196 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:17:05.414269 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.414272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.414410 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.415182 (+ 772us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.415374 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.416748 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.418177 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:05.418844 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:05.421902 (+ 3058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:05.435112 (+ 13210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:05.435116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:05.435120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:05.435156 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:05.441222 (+ 6066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.442193 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.442198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.442200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.442501 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:05.443143 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:05.443146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:05.443149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:05.443775 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:05.447978 (+ 4203us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:05.448131 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 06:17:05.448465 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12215,"thread_start_us":139,"threads_started":1}
I20260807 06:17:05.452956 9710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: Bootstrap starting.
I20260807 06:17:05.456712 9710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:05.457950 9710 log.cc:824] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:05.458675 9873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276: Generated new TSK 0
I20260807 06:17:05.460605 9873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 06:17:05.473917 9710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: No bootstrap required, opened a new log
I20260807 06:17:05.479288 9710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.479652 9710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcf06b5cbe048c28f50e7660a0441dc, State: Initialized, Role: FOLLOWER
I20260807 06:17:05.480122 9710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } }
I20260807 06:17:05.480806 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.481129 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:05.481999 9710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:05.482724 9671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2dcf06b5cbe048c28f50e7660a0441dc"
I20260807 06:17:05.483094 9671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:05.485330 9671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc91e2fbabb4e7886918c61ddbb7276 in term 1.
I20260807 06:17:05.486783 9670 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [term 1 FOLLOWER]: Refusing update from remote peer 8dc91e2fbabb4e7886918c61ddbb7276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:17:05.487717 9847 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:17:05.498979 9611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.495588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.9.133.190:39065)
0807 06:17:05.495752 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.495758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.497339 (+ 1581us) server_negotiation.cc:407] Connection header received
0807 06:17:05.497626 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.497630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.497703 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.497769 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.498786 (+ 1017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41716: BlockingRecv error: recv got EOF from 127.0.0.1:41716 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 06:17:05.501924 9917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.502125 9917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:17:05.503820 9906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:17:05.503921 9906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:17:05.504077 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dc91e2fbabb4e7886918c61ddbb7276. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:05.504199 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc [sys.catalog]: This master's current role is: FOLLOWER
W20260807 06:17:05.504397 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.496597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.9.133.189:34677)
0807 06:17:05.496721 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.496726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.499715 (+ 2989us) server_negotiation.cc:407] Connection header received
0807 06:17:05.499771 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.499775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.499840 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.499927 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.504255 (+ 4328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53312: BlockingRecv error: recv got EOF from 127.0.0.1:53312 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 06:17:05.505240 9878 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.499618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.9.133.188:40355)
0807 06:17:05.500154 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.500159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.500175 (+ 16us) server_negotiation.cc:407] Connection header received
0807 06:17:05.500229 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.500233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.500292 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.500387 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.505113 (+ 4726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46236: BlockingRecv error: recv got EOF from 127.0.0.1:46236 (error 108)
Metrics: {"server-negotiator.queue_time_us":419}
W20260807 06:17:05.509869 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.507983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.9.133.189:34677)
0807 06:17:05.508129 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.508134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.508654 (+ 520us) server_negotiation.cc:407] Connection header received
0807 06:17:05.508915 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.508919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.509029 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.509095 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.509739 (+ 644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53318: BlockingRecv error: recv got EOF from 127.0.0.1:53318 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 06:17:05.513046 9911 mvcc.cc:205] Tried to move back new op lower bound from 7315797710664171520 to 7315797710364123136. Current Snapshot: MvccSnapshot[applied={T|T < 7315797710413905920}]
W20260807 06:17:05.520536 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.514361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.9.133.189:34677)
0807 06:17:05.514511 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.514516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.518590 (+ 4074us) server_negotiation.cc:407] Connection header received
0807 06:17:05.519457 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.519461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.519537 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.519606 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.520378 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53334: BlockingRecv error: recv got EOF from 127.0.0.1:53334 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 06:17:05.527832 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.525824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.9.133.189:34677)
0807 06:17:05.525967 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.525971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.526639 (+ 668us) server_negotiation.cc:407] Connection header received
0807 06:17:05.526859 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.526863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.526926 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.527029 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.527691 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53340: BlockingRecv error: recv got EOF from 127.0.0.1:53340 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 06:17:05.536868 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.534725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53342 (local address 127.9.133.189:34677)
0807 06:17:05.534865 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.534870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.535616 (+ 746us) server_negotiation.cc:407] Connection header received
0807 06:17:05.535870 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.535875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.535959 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.536027 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.536712 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53342: BlockingRecv error: recv got EOF from 127.0.0.1:53342 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 06:17:05.553241 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.548177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.9.133.189:34677)
0807 06:17:05.548334 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.548339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.549125 (+ 786us) server_negotiation.cc:407] Connection header received
0807 06:17:05.549389 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.549394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.549465 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.549531 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.553068 (+ 3537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53358: BlockingRecv error: recv got EOF from 127.0.0.1:53358 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 06:17:05.575567 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.572858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.9.133.189:34677)
0807 06:17:05.573064 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.573071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.573982 (+ 911us) server_negotiation.cc:407] Connection header received
0807 06:17:05.574289 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.574295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.574367 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.574432 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.575395 (+ 963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53364: BlockingRecv error: recv got EOF from 127.0.0.1:53364 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260807 06:17:05.613966 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.611438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.9.133.189:34677)
0807 06:17:05.611624 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.611630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.612534 (+ 904us) server_negotiation.cc:407] Connection header received
0807 06:17:05.612841 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.612846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.612922 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.613002 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.613777 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53368: BlockingRecv error: recv got EOF from 127.0.0.1:53368 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 06:17:05.683743 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.681755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.9.133.189:34677)
0807 06:17:05.681970 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.681976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.682312 (+ 336us) server_negotiation.cc:407] Connection header received
0807 06:17:05.682514 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.682519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.682608 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.682772 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.683570 (+ 798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53380: BlockingRecv error: recv got EOF from 127.0.0.1:53380 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 06:17:05.817021 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:05.815110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.9.133.189:34677)
0807 06:17:05.815368 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:05.815373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:05.815539 (+ 166us) server_negotiation.cc:407] Connection header received
0807 06:17:05.815780 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:05.815785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:05.815859 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:05.816030 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:05.816806 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53392: BlockingRecv error: recv got EOF from 127.0.0.1:53392 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 06:17:06.038866 9847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc91e2fbabb4e7886918c61ddbb7276" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 39065 } } peers { permanent_uuid: "2dcf06b5cbe048c28f50e7660a0441dc" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 34677 } } peers { permanent_uuid: "60db51a36da54f798f6e083a551857d3" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 40355 } } }
I20260807 06:17:06.039237 9847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc91e2fbabb4e7886918c61ddbb7276 [sys.catalog]: This master's current role is: LEADER
W20260807 06:17:06.078132 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:06.076533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.9.133.189:34677)
0807 06:17:06.076724 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:06.076728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:06.076805 (+ 77us) server_negotiation.cc:407] Connection header received
0807 06:17:06.077045 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:06.077049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:06.077118 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:06.077217 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:06.077931 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53404: BlockingRecv error: recv got EOF from 127.0.0.1:53404 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 06:17:06.349948 9842 catalog_manager.cc:1259] Loaded cluster ID: e6aae1d4ef2a45a19180318a8d52af37
I20260807 06:17:06.350070 9842 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: loading cluster ID for follower catalog manager: success
I20260807 06:17:06.352231 9842 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: acquiring CA information for follower catalog manager: success
I20260807 06:17:06.353695 9842 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60db51a36da54f798f6e083a551857d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:17:06.505890 9906 catalog_manager.cc:1259] Loaded cluster ID: e6aae1d4ef2a45a19180318a8d52af37
I20260807 06:17:06.506038 9906 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: loading cluster ID for follower catalog manager: success
I20260807 06:17:06.508332 9906 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: acquiring CA information for follower catalog manager: success
I20260807 06:17:06.509922 9906 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2dcf06b5cbe048c28f50e7660a0441dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:17:06.598239 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:06.593196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.9.133.189:34677)
0807 06:17:06.593566 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:06.593571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:06.593593 (+ 22us) server_negotiation.cc:407] Connection header received
0807 06:17:06.593667 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:06.593671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:06.593854 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:06.593996 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:06.594336 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.595061 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.596182 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.596471 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.596569 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:06.596952 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:06.597566 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:06.597682 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:06.597827 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 06:17:06.597982 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":126,"threads_started":1}
I20260807 06:17:06.607501 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:06.602724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.9.133.188:40355)
0807 06:17:06.603144 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:06.603148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:06.603165 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:17:06.603278 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:06.603281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:06.603449 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:06.603579 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:06.603857 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.604539 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.605501 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.605763 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.605858 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:06.606200 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:06.606914 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:06.607117 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:06.607174 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 06:17:06.607241 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":139,"threads_started":1}
I20260807 06:17:06.616014 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:06.611531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46254 (local address 127.9.133.188:40355)
0807 06:17:06.611683 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:06.611688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:06.612028 (+ 340us) server_negotiation.cc:407] Connection header received
0807 06:17:06.612108 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:06.612116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:06.612198 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:06.612297 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:06.612616 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.613457 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.614617 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.614868 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.615080 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:06.615362 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:06.615485 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:06.615629 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:06.615752 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 06:17:06.615816 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 06:17:06.616349 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:06.611798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.9.133.189:34677)
0807 06:17:06.611983 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:06.611989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:06.612328 (+ 339us) server_negotiation.cc:407] Connection header received
0807 06:17:06.612538 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:06.612545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:06.612647 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:06.612745 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:06.613015 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.613747 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.614972 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.615364 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.615487 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:06.615757 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:06.615854 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:06.615904 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:06.616118 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 06:17:06.616202 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 06:17:06.617365 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:06.611391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.9.133.190:39065)
0807 06:17:06.612225 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:06.612230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:06.612248 (+ 18us) server_negotiation.cc:407] Connection header received
0807 06:17:06.612392 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:06.612395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:06.612638 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:06.612794 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:06.613163 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.613969 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.615344 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:06.615734 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:06.615826 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:06.616387 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:06.616905 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:06.617014 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:06.617100 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 06:17:06.617171 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":100,"threads_started":1}
I20260807 06:17:06.627966 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9511
I20260807 06:17:06.645187 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9617
I20260807 06:17:06.663524 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 9716
2026-08-07T06:17:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3618 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 06:17:06 dist-test-slave-h3ft krb5kdc[10025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 06:17:06 dist-test-slave-h3ft krb5kdc[10025](info): set up 2 sockets
Aug 07 06:17:06 dist-test-slave-h3ft krb5kdc[10025](info): commencing operation
krb5kdc: starting...
W20260807 06:17:07.967743 9750 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/test-user.keytab.
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T06:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T06:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.133.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
Entry for principal kudu/127.9.133.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab.
I20260807 06:17:08.135210 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35139
--webserver_interface=127.9.133.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:35139,127.9.133.189:32777,127.9.133.188:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 06:17:08.274684 10043 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:17:08.275146 10043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:08.275226 10043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:08.275309 10043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:08.279623 10043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:08.279726 10043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:08.279762 10043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:08.279803 10043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:08.279840 10043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:08.285594 10043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:35139,127.9.133.189:32777,127.9.133.188:37161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.190.keytab
--principal=kudu/127.9.133.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.190:35139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.133.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:08.287075 10043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:08.288396 10043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:08.298792 10053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:08.298816 10050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:08.298877 10043 server_base.cc:1034] running on GCE node
W20260807 06:17:08.298821 10051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:08.299875 10043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:08.301203 10043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:08.302388 10043 hybrid_clock.cc:648] HybridClock initialized: now 1786083428302369 us; error 34 us; skew 500 ppm
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:08.306213 10043 init.cc:377] Logged in from keytab as kudu/127.9.133.190@KRBTEST.COM (short username kudu)
I20260807 06:17:08.307390 10043 webserver.cc:460] Webserver started at http://127.9.133.190:44451/ using document root <none> and password file <none>
I20260807 06:17:08.308094 10043 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:08.308194 10043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:08.308470 10043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:08.310974 10043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data/instance:
uuid: "e153aa15f7424339bf7504f30a6f1090"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "0c0fd73ff1eaf35ed3b82c98b4d333d8"
server_key_iv: "08f504a5597ed8fb4f0852709e0134ee"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.311694 10043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal/instance:
uuid: "e153aa15f7424339bf7504f30a6f1090"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "0c0fd73ff1eaf35ed3b82c98b4d333d8"
server_key_iv: "08f504a5597ed8fb4f0852709e0134ee"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.316387 10043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 06:17:08.319568 10062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:08.321210 10043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 06:17:08.321398 10043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal
uuid: "e153aa15f7424339bf7504f30a6f1090"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "0c0fd73ff1eaf35ed3b82c98b4d333d8"
server_key_iv: "08f504a5597ed8fb4f0852709e0134ee"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.321538 10043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:08.338155 10043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:08.341778 10043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:08.342032 10043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:08.350865 10043 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.190:35139
I20260807 06:17:08.351272 10132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.190:35139 every 8 connection(s)
I20260807 06:17:08.352108 10043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/data/info.pb
I20260807 06:17:08.355744 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10043
I20260807 06:17:08.355835 10134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:08.355902 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-0/wal/instance
I20260807 06:17:08.356331 9750 external_mini_cluster.cc:1428] Setting key 2625fd15dbc0d974f99206b29ef919f2
I20260807 06:17:08.361532 10134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.133.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.189@KRBTEST.COM" created.
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:17:08.380640 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.366046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.9.133.190:35139)
0807 06:17:08.366397 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.366409 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.366463 (+ 54us) server_negotiation.cc:407] Connection header received
0807 06:17:08.367069 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.367074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.367301 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.367537 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.368538 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.369666 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.370743 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.371297 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.371583 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.374854 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.374860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.374874 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.374898 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.376815 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.377497 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.377502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.377503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.377677 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.378105 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.378107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.378108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.378577 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.378689 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.379329 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.379592 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":233,"thread_start_us":114,"threads_started":1}
I20260807 06:17:08.380646 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.364790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35139 (local address 127.0.0.1:46790)
0807 06:17:08.365491 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.365552 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.366660 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.367524 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.367530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.368045 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.368344 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.368360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.369813 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.369817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.370615 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.370624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.371490 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.371495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.371690 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.372506 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.372532 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.374671 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.376951 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.376965 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.376981 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.377368 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.377810 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.377817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.377821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.378003 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.378728 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.378739 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.378970 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.379252 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.379706 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":94,"thread_start_us":198,"threads_started":1}
I20260807 06:17:08.383946 10134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:08.384953 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.384618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:52910)
0807 06:17:08.384783 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.384878 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 06:17:08.385295 10068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
I20260807 06:17:08.386313 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.386085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:52918)
0807 06:17:08.386203 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.386257 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 06:17:08.386524 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:32777: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab.
Entry for principal kudu/127.9.133.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab.
I20260807 06:17:08.392436 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:32777
--webserver_interface=127.9.133.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab
--principal=kudu/127.9.133.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:35139,127.9.133.189:32777,127.9.133.188:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:17:08.435873 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } attempt: 1
I20260807 06:17:08.437148 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.436750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:52932)
0807 06:17:08.436952 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.437037 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 06:17:08.438026 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.437808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:52938)
0807 06:17:08.437913 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.437962 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 06:17:08.438208 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:32777: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
I20260807 06:17:08.506438 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } attempt: 2
I20260807 06:17:08.507717 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.507314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:52940)
0807 06:17:08.507513 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.507611 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 06:17:08.508646 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.508437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:52956)
0807 06:17:08.508544 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.508589 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 06:17:08.508872 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:32777: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
W20260807 06:17:08.530841 10143 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:17:08.531260 10143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:08.531322 10143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:08.531427 10143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:08.535593 10143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:08.535674 10143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:08.535694 10143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:08.535713 10143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:08.535729 10143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:08.541635 10143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:35139,127.9.133.189:32777,127.9.133.188:37161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.189.keytab
--principal=kudu/127.9.133.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.189:32777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.133.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:08.543109 10143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:08.544486 10143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 06:17:08.555193 10152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:08.555193 10150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:08.555229 10149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:08.555440 10143 server_base.cc:1034] running on GCE node
I20260807 06:17:08.556232 10143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:08.557588 10143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:08.558794 10143 hybrid_clock.cc:648] HybridClock initialized: now 1786083428558770 us; error 38 us; skew 500 ppm
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:08.562191 10143 init.cc:377] Logged in from keytab as kudu/127.9.133.189@KRBTEST.COM (short username kudu)
I20260807 06:17:08.563331 10143 webserver.cc:460] Webserver started at http://127.9.133.189:44703/ using document root <none> and password file <none>
I20260807 06:17:08.563971 10143 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:08.564034 10143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:08.564230 10143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:08.566630 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data/instance:
uuid: "4fa8334ae385409ea5f2bd96e24d875d"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "55e391fd8686fb998494589825b0ebed"
server_key_iv: "b2e2683de344c63a3d6891ebce51d5d4"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.567301 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal/instance:
uuid: "4fa8334ae385409ea5f2bd96e24d875d"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "55e391fd8686fb998494589825b0ebed"
server_key_iv: "b2e2683de344c63a3d6891ebce51d5d4"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.571686 10143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 06:17:08.574538 10163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:08.576056 10143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 06:17:08.576170 10143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal
uuid: "4fa8334ae385409ea5f2bd96e24d875d"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "55e391fd8686fb998494589825b0ebed"
server_key_iv: "b2e2683de344c63a3d6891ebce51d5d4"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.576268 10143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:08.594589 10143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:08.598345 10143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:08.598539 10143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:08.600047 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } attempt: 3
I20260807 06:17:08.601102 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.600753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:51312)
0807 06:17:08.600926 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.600999 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 06:17:08.601961 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.601702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:51322)
0807 06:17:08.601848 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.601896 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 06:17:08.602102 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.189:32777: Network error: Client connection negotiation failed: client connection to 127.9.133.189:32777: connect: Connection refused (error 111)
I20260807 06:17:08.607744 10143 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.189:32777
I20260807 06:17:08.608088 10233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.189:32777 every 8 connection(s)
I20260807 06:17:08.609120 10143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/data/info.pb
I20260807 06:17:08.612236 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10143
I20260807 06:17:08.612428 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-1/wal/instance
I20260807 06:17:08.612793 9750 external_mini_cluster.cc:1428] Setting key 7fc9bbd7acacd1b3aebe72b20f9ac1c7
I20260807 06:17:08.613060 10235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:08.617822 10235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.133.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.133.188@KRBTEST.COM" created.
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:17:08.634971 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.620424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.9.133.190:35139)
0807 06:17:08.620637 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.620641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.621919 (+ 1278us) server_negotiation.cc:407] Connection header received
0807 06:17:08.623173 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.623176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.623257 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.623374 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.624819 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.625448 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.626922 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.627270 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.627397 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.630257 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.630261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.630263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.630295 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.632305 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.632974 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.632979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.632982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.633087 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.633518 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.633523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.633525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.633769 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.633910 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.634574 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.634736 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 06:17:08.636121 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.621040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35139 (local address 127.0.0.1:56842)
0807 06:17:08.621689 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.621742 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.622859 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.623637 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.623645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.624047 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.624646 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.624662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.625615 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.625618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.626797 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.626808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.627449 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.627454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.627675 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.628414 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.628432 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.630054 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.632472 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.632480 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.632501 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.632846 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.633209 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.633213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.633215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.633404 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.633899 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.633907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.634255 (+ 348us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.634506 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.634942 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":164,"threads_started":1}
I20260807 06:17:08.638031 10235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.133.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab.
Entry for principal kudu/127.9.133.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab.
I20260807 06:17:08.647435 9750 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
/tmp/dist-test-taskrsqWAA/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:37161
--webserver_interface=127.9.133.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab
--principal=kudu/127.9.133.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.133.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.133.190:35139,127.9.133.189:32777,127.9.133.188:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 06:17:08.650141 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.638666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:51330)
0807 06:17:08.638967 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.638984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.639142 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.639781 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.639785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.640015 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.640116 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.640122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.641244 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.641251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.641887 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.641896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.642605 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.642608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.642729 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.643414 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.643428 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.645213 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.647841 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.647848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.647854 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.648218 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.648583 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.648590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.648595 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.648768 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.649364 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.649370 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.649513 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.649650 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.649860 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 06:17:08.650396 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.638814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.9.133.189:32777)
0807 06:17:08.639346 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.639357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.639403 (+ 46us) server_negotiation.cc:407] Connection header received
0807 06:17:08.639441 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.639447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.639634 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.639812 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.640241 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.641114 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.642018 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.642462 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.642601 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.645419 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.645439 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.645461 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.645486 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.647697 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.648369 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.648375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.648376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.648468 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.648903 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.648907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.648908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.649235 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.649378 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.649782 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.650078 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":286,"threads_started":1}
I20260807 06:17:08.652765 10235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:08.653683 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.653385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51094)
0807 06:17:08.653534 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.653617 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 06:17:08.653928 10167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
I20260807 06:17:08.654915 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.654678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51100)
0807 06:17:08.654793 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.654852 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 06:17:08.655115 10235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
I20260807 06:17:08.704437 10235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 1
I20260807 06:17:08.705737 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.705319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51104)
0807 06:17:08.705542 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.705624 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 06:17:08.706665 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.706385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51112)
0807 06:17:08.706534 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.706595 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 06:17:08.706810 10235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
I20260807 06:17:08.745316 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } attempt: 4
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
I20260807 06:17:08.758210 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.746403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.9.133.189:32777)
0807 06:17:08.746629 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.746635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.746812 (+ 177us) server_negotiation.cc:407] Connection header received
0807 06:17:08.747181 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.747188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.747308 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.747468 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.748169 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.749242 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.750865 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.751257 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.751393 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.754118 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.754122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.754124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.754157 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.756066 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.756719 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.756723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.756725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.756821 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.757192 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.757195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.757196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.757440 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.757545 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.757901 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.758043 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 06:17:08.758245 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.746323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:51346)
0807 06:17:08.746701 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.746718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.746897 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.747521 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.747524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.747800 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.747997 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.748005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.749404 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.749408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.750653 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.750667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.751419 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.751424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.751569 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.752296 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.752310 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.753922 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.756227 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.756231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.756234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.756577 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.756939 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.756943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.756946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.757066 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.757561 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.757564 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.757706 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.757835 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.758033 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260807 06:17:08.759378 10134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } has no permanent_uuid. Determining permanent_uuid...
I20260807 06:17:08.760316 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.759997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51120)
0807 06:17:08.760156 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.760225 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 06:17:08.761127 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.760877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51128)
0807 06:17:08.760995 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.761055 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 06:17:08.761279 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
I20260807 06:17:08.775506 10235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 2
I20260807 06:17:08.776829 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.776255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51132)
0807 06:17:08.776598 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.776692 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
I20260807 06:17:08.779672 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.779314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51144)
0807 06:17:08.779474 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.779552 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 06:17:08.779860 10235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
W20260807 06:17:08.800843 10243 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 06:17:08.801317 10243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 06:17:08.801385 10243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 06:17:08.801470 10243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 06:17:08.805850 10243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 06:17:08.805955 10243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 06:17:08.805994 10243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 06:17:08.806052 10243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 06:17:08.806107 10243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 06:17:08.812103 10243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.133.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.133.190:35139,127.9.133.189:32777,127.9.133.188:37161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/krb5kdc/kudu_127.9.133.188.keytab
--principal=kudu/127.9.133.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.133.188:37161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.133.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 05:39:39 UTC on ccycloud.quasar-xaluan.root.comops.site
build id 4571902
I20260807 06:17:08.813544 10243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 06:17:08.814919 10243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 06:17:08.820442 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 1
I20260807 06:17:08.821501 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.821136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51148)
0807 06:17:08.821309 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.821385 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 06:17:08.822453 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.822151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51162)
0807 06:17:08.822313 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.822383 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 06:17:08.822620 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
W20260807 06:17:08.825954 10256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:08.826288 10243 server_base.cc:1034] running on GCE node
W20260807 06:17:08.825991 10251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 06:17:08.825995 10253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 06:17:08.827235 10243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 06:17:08.828805 10243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 06:17:08.830044 10243 hybrid_clock.cc:648] HybridClock initialized: now 1786083428830023 us; error 34 us; skew 500 ppm
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 06:17:08.833779 10243 init.cc:377] Logged in from keytab as kudu/127.9.133.188@KRBTEST.COM (short username kudu)
I20260807 06:17:08.834990 10243 webserver.cc:460] Webserver started at http://127.9.133.188:39325/ using document root <none> and password file <none>
I20260807 06:17:08.835772 10243 fs_manager.cc:359] Metadata directory not provided
I20260807 06:17:08.835876 10243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 06:17:08.836148 10243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 06:17:08.838841 10243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data/instance:
uuid: "bb7a240cc10f409aafe3866f02cbce86"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "064df85c18739c74d31bd98e009390c0"
server_key_iv: "f51ae6a672deaa43116cab2287e71057"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.839592 10243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal/instance:
uuid: "bb7a240cc10f409aafe3866f02cbce86"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "064df85c18739c74d31bd98e009390c0"
server_key_iv: "f51ae6a672deaa43116cab2287e71057"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.845516 10243 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 06:17:08.849591 10268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 06:17:08.851481 10243 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 06:17:08.851617 10243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data,/tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal
uuid: "bb7a240cc10f409aafe3866f02cbce86"
format_stamp: "Formatted at 2026-08-07 06:17:08 on dist-test-slave-h3ft"
server_key: "064df85c18739c74d31bd98e009390c0"
server_key_iv: "f51ae6a672deaa43116cab2287e71057"
server_key_version: "encryptionkey@0"
I20260807 06:17:08.851744 10243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 06:17:08.871145 10235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 3
I20260807 06:17:08.872433 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.872050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51168)
0807 06:17:08.872243 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.872324 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 06:17:08.873390 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.873081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51172)
0807 06:17:08.873267 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.873321 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260807 06:17:08.873541 10235 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
I20260807 06:17:08.883744 10243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 06:17:08.887692 10243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 06:17:08.887953 10243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 06:17:08.889808 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 2
I20260807 06:17:08.890784 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.890439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51176)
0807 06:17:08.890605 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.890680 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 06:17:08.891834 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.891523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51190)
0807 06:17:08.891661 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.891751 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 06:17:08.892022 10134 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.133.188:37161: Network error: Client connection negotiation failed: client connection to 127.9.133.188:37161: connect: Connection refused (error 111)
I20260807 06:17:08.897076 10243 rpc_server.cc:307] RPC server started. Bound to: 127.9.133.188:37161
I20260807 06:17:08.897419 10343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.133.188:37161 every 8 connection(s)
I20260807 06:17:08.898360 10243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/data/info.pb
I20260807 06:17:08.899057 9750 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu as pid 10243
I20260807 06:17:08.899248 9750 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrsqWAA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786083100784614-9750-0/minicluster-data/master-2/wal/instance
I20260807 06:17:08.899642 9750 external_mini_cluster.cc:1428] Setting key 2c67d2763259b65ef931f3a42ab9baea
I20260807 06:17:08.902431 10344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 06:17:08.908120 10344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
I20260807 06:17:08.918318 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.901472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.9.133.188:37161)
0807 06:17:08.902007 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.902035 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.902098 (+ 63us) server_negotiation.cc:407] Connection header received
0807 06:17:08.902987 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.903068 (+ 81us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.903553 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.903931 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.905112 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.906362 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.907709 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.908315 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.908476 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.912096 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.912102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.912113 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.912153 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.914120 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.914644 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.914650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.914655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.914785 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.915186 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.915191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.915194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.915861 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.916114 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.916541 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.916949 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":184,"threads_started":1}
I20260807 06:17:08.921543 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.910781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.9.133.190:35139)
0807 06:17:08.911051 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.911057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.911463 (+ 406us) server_negotiation.cc:407] Connection header received
0807 06:17:08.911733 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.911735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.911792 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.911869 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.912785 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.913331 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.914247 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.914522 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.914645 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.917397 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.917401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.917404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.917440 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.919340 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.919944 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.919949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.919951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.920055 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.920378 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.920381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.920383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.920663 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.920776 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.921197 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.921324 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 06:17:08.921974 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.910704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.190:35139 (local address 127.0.0.1:56854)
0807 06:17:08.911306 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.911376 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.911598 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.911968 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.911975 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.912464 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.912637 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.912644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.913458 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.913461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.914111 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.914119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.914646 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.914649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.914785 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.915333 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.915347 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.917233 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.919457 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.919466 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.919472 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.919794 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.920178 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.920180 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.920182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.920279 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.920805 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.920814 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.920975 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.921134 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.921342 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":210,"threads_started":1}
I20260807 06:17:08.923635 10344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.190@KRBTEST.COM
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
I20260807 06:17:08.933115 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.921762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.9.133.190:35139)
0807 06:17:08.922210 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.922226 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.922239 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:17:08.922279 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.922281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.922320 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.922438 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.923291 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.924084 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.925043 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.925327 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.925460 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.928553 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.928557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.928559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.928587 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.930603 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.931169 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.931175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.931180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.931302 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.931685 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.931688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.931689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.932061 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.932401 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.932639 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.932809 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":195,"threads_started":1}
I20260807 06:17:08.933707 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.924426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.9.133.189:32777)
0807 06:17:08.924613 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.924617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.924630 (+ 13us) server_negotiation.cc:407] Connection header received
0807 06:17:08.924675 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.924677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.924722 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.924814 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.925438 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.926020 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.926888 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.927183 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.927300 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.929498 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.929506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.929509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.929540 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.931313 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.932016 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.932020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.932022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.932136 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.932541 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.932548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.932549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.932845 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.932969 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.933432 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.933563 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:17:08.934257 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.924273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.189:32777 (local address 127.0.0.1:51350)
0807 06:17:08.924392 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.924407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.924510 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.924830 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.924833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.925113 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.925272 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.925279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.926140 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.926145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.926745 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.926754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.927302 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.927306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.927423 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.928004 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.928020 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.929329 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.931644 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.931647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.931648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.931858 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.932259 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.932263 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.932265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.932390 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.933032 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.933035 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.933166 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.933918 (+ 752us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.934112 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 06:17:08.935249 10344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.133.189@KRBTEST.COM
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.188@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:17:08.946035 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.934788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.9.133.189:32777)
0807 06:17:08.934953 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.934958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.935191 (+ 233us) server_negotiation.cc:407] Connection header received
0807 06:17:08.935224 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.935226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.935268 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.935354 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.936213 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.936994 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.938199 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.938546 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.938680 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.942008 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.942016 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.942019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.942056 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.943737 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.944307 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.944311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.944313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.944418 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.944733 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.944736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.944737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.945061 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.945305 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.945471 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.945620 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 06:17:08.946034 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.935947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51208)
0807 06:17:08.936182 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.936197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.936281 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.936507 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.936510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.936693 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.936785 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.936792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.938373 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.938377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.939215 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.939236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.939973 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.939976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.940043 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.940576 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.940590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.942106 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:08.944307 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.944310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.944313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.944602 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.944891 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:08.944894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:08.944896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:08.944984 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:08.945433 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:08.945436 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:08.945556 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 06:17:08.945673 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 06:17:08.945825 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260807 06:17:08.946034 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.936061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.9.133.188:37161)
0807 06:17:08.936190 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.936193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.936237 (+ 44us) server_negotiation.cc:407] Connection header received
0807 06:17:08.936358 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.936360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.936395 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.936465 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.936893 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.938228 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.939373 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.939811 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.939942 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.942265 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.942268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.942270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.942292 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:08.944201 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.944701 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.944706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.944708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.944797 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:08.945079 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:08.945081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:08.945082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:08.945325 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:08.945437 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:08.945757 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 06:17:08.945892 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":47}
I20260807 06:17:08.950768 9750 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 06:17:08.950845 10344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: Bootstrap starting.
I20260807 06:17:08.952453 10099 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e153aa15f7424339bf7504f30a6f1090"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56868
I20260807 06:17:08.954782 10344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:08.955978 10344 log.cc:824] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:08.958777 10344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: No bootstrap required, opened a new log
I20260807 06:17:08.962795 10344 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:08.963229 10344 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb7a240cc10f409aafe3866f02cbce86, State: Initialized, Role: FOLLOWER
I20260807 06:17:08.963685 10344 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:08.964442 10356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:08.964633 10356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:08.965456 10344 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 06:17:08.971922 10371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:17:08.972030 10371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:17:08.970487 10243 master_runner.cc:186] Error getting master registration for 127.9.133.190:35139: OK, instance_id { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" instance_seqno: 1786083428348448 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:17:08.974471 10243 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:17:08.992240 10134 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 3
Aug 07 06:17:08 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.190@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:17:09.003487 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.993172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51224)
0807 06:17:08.993378 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:08.993399 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:08.993551 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:08.993883 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:08.993886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:08.994170 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:08.994312 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.994319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.995229 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.995234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.995821 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:08.995830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.996349 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.996352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:08.996441 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.997149 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:08.997182 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:08.998925 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:09.001370 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:09.001374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:09.001378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:09.001705 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:09.002051 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:09.002054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:09.002056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:09.002158 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:09.002786 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:09.002792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:09.002922 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 06:17:09.003066 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 06:17:09.003249 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 06:17:09.003530 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:08.993271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.9.133.188:37161)
0807 06:17:08.993425 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:08.993432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:08.993457 (+ 25us) server_negotiation.cc:407] Connection header received
0807 06:17:08.993678 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:08.993681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:08.993752 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:08.993860 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:08.994464 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.995099 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.995961 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:08.996211 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:08.996339 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:08.999144 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:08.999150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:08.999156 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:08.999194 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:09.001214 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.001819 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.001823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.001826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.001940 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.002260 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.002264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.002266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.002678 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:09.002795 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:09.003162 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 06:17:09.003320 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 06:17:09.008466 10134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090: Bootstrap starting.
I20260807 06:17:09.011714 10134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:09.012676 10134 log.cc:824] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:09.015224 10134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090: No bootstrap required, opened a new log
I20260807 06:17:09.016706 10235 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } attempt: 4
I20260807 06:17:09.018695 10134 raft_consensus.cc:348] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:09.019025 10134 raft_consensus.cc:729] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e153aa15f7424339bf7504f30a6f1090, State: Initialized, Role: FOLLOWER
I20260807 06:17:09.019457 10134 consensus_queue.cc:260] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:09.020147 10374 sys_catalog.cc:455] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.020435 10374 sys_catalog.cc:458] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:09.022058 10134 sys_catalog.cc:564] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:09.022107 10099 raft_consensus.cc:482] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 06:17:09.022229 10099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:09.024099 10099 raft_consensus.cc:504] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
Aug 07 06:17:09 dist-test-slave-h3ft krb5kdc[10025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786083428, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.133.189@KRBTEST.COM for kudu/127.9.133.188@KRBTEST.COM
I20260807 06:17:09.027309 10307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e153aa15f7424339bf7504f30a6f1090" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb7a240cc10f409aafe3866f02cbce86"
I20260807 06:17:09.027698 10307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:09.029605 10307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e153aa15f7424339bf7504f30a6f1090 in term 1.
I20260807 06:17:09.029944 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:09.017438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.133.188:37161 (local address 127.0.0.1:51230)
0807 06:17:09.017784 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 06:17:09.017804 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 06:17:09.017930 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 06:17:09.018448 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 06:17:09.018452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 06:17:09.018730 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 06:17:09.018879 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:09.018887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.020232 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.020237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:09.021254 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 06:17:09.021268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.021986 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.021990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 06:17:09.022101 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:09.022834 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 06:17:09.022850 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 06:17:09.024712 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 06:17:09.027728 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:09.027733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:09.027735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:09.028060 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:09.028563 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 06:17:09.028567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 06:17:09.028569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 06:17:09.028682 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 06:17:09.029248 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 06:17:09.029251 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 06:17:09.029392 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 06:17:09.029526 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 06:17:09.029735 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260807 06:17:09.030047 10099 leader_election.cc:288] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [CANDIDATE]: Term 1 election: Requested vote from peers 4fa8334ae385409ea5f2bd96e24d875d (127.9.133.189:32777), bb7a240cc10f409aafe3866f02cbce86 (127.9.133.188:37161)
I20260807 06:17:09.030124 10065 leader_election.cc:302] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb7a240cc10f409aafe3866f02cbce86, e153aa15f7424339bf7504f30a6f1090; no voters:
I20260807 06:17:09.030568 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:09.017555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.9.133.188:37161)
0807 06:17:09.018145 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.018151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.018168 (+ 17us) server_negotiation.cc:407] Connection header received
0807 06:17:09.018220 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.018224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.018281 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.018366 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.019106 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.020064 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.021446 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.021837 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.022555 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:09.024917 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:09.024922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:09.024926 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:09.024966 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:09.027452 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.028359 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.028363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.028365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.028467 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.028814 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.028817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.028819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.029083 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:09.029195 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:09.030194 (+ 999us) server_negotiation.cc:299] Negotiation successful
0807 06:17:09.030378 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476}
W20260807 06:17:09.033743 10390 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:17:09.034013 10390 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:17:09.033849 10043 master_runner.cc:186] Error getting master registration for 127.9.133.189:32777: OK, instance_id { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" instance_seqno: 1786083428605250 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 06:17:09.037655 10235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: Bootstrap starting.
I20260807 06:17:09.037740 10043 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 06:17:09.039247 10374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 06:17:09.039598 10374 raft_consensus.cc:686] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [term 1 LEADER]: Becoming Leader. State: Replica: e153aa15f7424339bf7504f30a6f1090, State: Running, Role: LEADER
I20260807 06:17:09.040066 10374 consensus_queue.cc:237] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:09.042306 10235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: Neither blocks nor log segments found. Creating new log.
I20260807 06:17:09.042526 10374 sys_catalog.cc:455] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e153aa15f7424339bf7504f30a6f1090. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.042719 10374 sys_catalog.cc:458] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:09.044798 10235 log.cc:824] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: Log is configured to *not* fsync() on all Append() calls
I20260807 06:17:09.046846 10401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 06:17:09.048211 10401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 06:17:09.058439 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:09.032804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.9.133.189:32777)
0807 06:17:09.032955 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.032961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.044757 (+ 11796us) server_negotiation.cc:407] Connection header received
0807 06:17:09.045120 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.045124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.045197 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.045261 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.046510 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.047311 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.049160 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.049485 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.049569 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:09.053050 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:09.053054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:09.053058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:09.053093 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:09.055451 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.056358 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.056362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.056364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.056474 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.057085 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.057089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.057091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.057352 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:09.057467 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:09.058078 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 06:17:09.058239 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 06:17:09.064793 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:09.043500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.9.133.190:35139)
0807 06:17:09.043683 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.043691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.051360 (+ 7669us) server_negotiation.cc:407] Connection header received
0807 06:17:09.051578 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.051581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.051644 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.051705 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.053436 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.054293 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.055579 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.055976 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.056059 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:09.058575 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:09.058582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:09.058586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:09.058628 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:09.060593 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.061161 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.061165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.061167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.061291 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.063590 (+ 2299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.063595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.063597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.063888 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:09.064010 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:09.064402 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 06:17:09.064593 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 06:17:09.065362 10307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [term 1 FOLLOWER]: Refusing update from remote peer e153aa15f7424339bf7504f30a6f1090: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:17:09.066731 10399 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 06:17:09.074193 10356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e153aa15f7424339bf7504f30a6f1090. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.074402 10356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:09.074980 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:09.044198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.9.133.188:37161)
0807 06:17:09.044355 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.044361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.052893 (+ 8532us) server_negotiation.cc:407] Connection header received
0807 06:17:09.053103 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.053107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.053174 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.053231 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.061213 (+ 7982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.062067 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.067450 (+ 5383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:09.067874 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:09.067967 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:09.070435 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 06:17:09.070440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 06:17:09.070444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 06:17:09.070499 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 06:17:09.072615 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.073315 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.073320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.073322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.073432 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 06:17:09.073775 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 06:17:09.073779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 06:17:09.073781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 06:17:09.074065 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 06:17:09.074195 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:09.074614 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 06:17:09.074782 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 06:17:09.079704 10235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: No bootstrap required, opened a new log
I20260807 06:17:09.086738 10399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.086745 10235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:09.087040 10399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:09.087097 10235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa8334ae385409ea5f2bd96e24d875d, State: Initialized, Role: FOLLOWER
I20260807 06:17:09.087553 10235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } }
I20260807 06:17:09.089512 10235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 06:17:09.089593 10197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e153aa15f7424339bf7504f30a6f1090" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4fa8334ae385409ea5f2bd96e24d875d"
I20260807 06:17:09.089838 10196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [term 0 FOLLOWER]: Advancing to term 1
I20260807 06:17:09.091123 10406 mvcc.cc:205] Tried to move back new op lower bound from 7315797725426860032 to 7315797725352792064. Current Snapshot: MvccSnapshot[applied={T|T < 7315797725426860032}]
I20260807 06:17:09.091744 10196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [term 1 FOLLOWER]: Refusing update from remote peer e153aa15f7424339bf7504f30a6f1090: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 06:17:09.094195 10197 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e153aa15f7424339bf7504f30a6f1090 in term 1.
I20260807 06:17:09.096009 10410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.096201 10410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:09.096570 10374 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 06:17:09.100484 10175 tablet.cc:2369] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 06:17:09.102519 10410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e153aa15f7424339bf7504f30a6f1090. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.102658 10410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:09.102741 10410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.102813 10410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:09.104125 10407 mvcc.cc:205] Tried to move back new op lower bound from 7315797725426860032 to 7315797725352792064. Current Snapshot: MvccSnapshot[applied={T|T < 7315797725426860032}]
I20260807 06:17:09.104897 10356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.105026 10356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 06:17:09.106729 10401 catalog_manager.cc:1347] Generated new cluster ID: 49ba8e6aa968430cbc51789423864a3c
I20260807 06:17:09.106827 10401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 06:17:09.111073 10425 mvcc.cc:205] Tried to move back new op lower bound from 7315797725426860032 to 7315797725352792064. Current Snapshot: MvccSnapshot[applied={T|T < 7315797725426860032}]
W20260807 06:17:09.115249 10423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 06:17:09.115392 10423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 06:17:09.119467 10374 sys_catalog.cc:455] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e153aa15f7424339bf7504f30a6f1090" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e153aa15f7424339bf7504f30a6f1090" member_type: VOTER last_known_addr { host: "127.9.133.190" port: 35139 } } peers { permanent_uuid: "4fa8334ae385409ea5f2bd96e24d875d" member_type: VOTER last_known_addr { host: "127.9.133.189" port: 32777 } } peers { permanent_uuid: "bb7a240cc10f409aafe3866f02cbce86" member_type: VOTER last_known_addr { host: "127.9.133.188" port: 37161 } } }
I20260807 06:17:09.119654 10374 sys_catalog.cc:458] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090 [sys.catalog]: This master's current role is: LEADER
I20260807 06:17:09.129021 10401 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 06:17:09.130261 10401 catalog_manager.cc:1504] Loading token signing keys...
I20260807 06:17:09.146489 10401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e153aa15f7424339bf7504f30a6f1090: Generated new TSK 0
I20260807 06:17:09.147851 10401 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 06:17:09.202859 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.200538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.9.133.189:32777)
0807 06:17:09.200713 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.200719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.201520 (+ 801us) server_negotiation.cc:407] Connection header received
0807 06:17:09.201767 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.201772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.201835 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.201895 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.202670 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51374: BlockingRecv error: recv got EOF from 127.0.0.1:51374 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 06:17:09.210060 10353 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.203645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.9.133.190:35139)
0807 06:17:09.205932 (+ 2287us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.205938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.205957 (+ 19us) server_negotiation.cc:407] Connection header received
0807 06:17:09.206015 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.206019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.206076 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.206165 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.209879 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56878: BlockingRecv error: recv got EOF from 127.0.0.1:56878 (error 108)
Metrics: {"server-negotiator.queue_time_us":2166}
W20260807 06:17:09.211525 10348 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.204221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.9.133.188:37161)
0807 06:17:09.204357 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.204362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.209930 (+ 5568us) server_negotiation.cc:407] Connection header received
0807 06:17:09.210615 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.210619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.210689 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.210746 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.211368 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51238: BlockingRecv error: recv got EOF from 127.0.0.1:51238 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 06:17:09.228981 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.227074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.9.133.189:32777)
0807 06:17:09.227239 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.227246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.227817 (+ 571us) server_negotiation.cc:407] Connection header received
0807 06:17:09.227949 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.227952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.228054 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.228139 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.228818 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51378: BlockingRecv error: recv got EOF from 127.0.0.1:51378 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 06:17:09.234622 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.233293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.9.133.189:32777)
0807 06:17:09.233432 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.233437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.233604 (+ 167us) server_negotiation.cc:407] Connection header received
0807 06:17:09.233770 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.233774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.233836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.233930 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.234478 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51390: BlockingRecv error: recv got EOF from 127.0.0.1:51390 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 06:17:09.239969 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.238546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.9.133.189:32777)
0807 06:17:09.238691 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.238695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.239028 (+ 333us) server_negotiation.cc:407] Connection header received
0807 06:17:09.239164 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.239168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.239227 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.239316 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.239838 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51404: BlockingRecv error: recv got EOF from 127.0.0.1:51404 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 06:17:09.247606 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.246148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.9.133.189:32777)
0807 06:17:09.246271 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.246275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.246544 (+ 269us) server_negotiation.cc:407] Connection header received
0807 06:17:09.246703 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.246707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.246764 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.246857 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.247463 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51420: BlockingRecv error: recv got EOF from 127.0.0.1:51420 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 06:17:09.259945 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.258396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.9.133.189:32777)
0807 06:17:09.258543 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.258548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.258814 (+ 266us) server_negotiation.cc:407] Connection header received
0807 06:17:09.258976 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.258980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.259080 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.259184 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.259790 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51430: BlockingRecv error: recv got EOF from 127.0.0.1:51430 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 06:17:09.280339 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.278622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.9.133.189:32777)
0807 06:17:09.278809 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.278814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.279058 (+ 244us) server_negotiation.cc:407] Connection header received
0807 06:17:09.279329 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.279333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.279393 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.279508 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.280141 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51440: BlockingRecv error: recv got EOF from 127.0.0.1:51440 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 06:17:09.317019 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.315415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.9.133.189:32777)
0807 06:17:09.315615 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.315621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.315827 (+ 206us) server_negotiation.cc:407] Connection header received
0807 06:17:09.316032 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.316035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.316094 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.316175 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.316847 (+ 672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51448: BlockingRecv error: recv got EOF from 127.0.0.1:51448 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 06:17:09.385578 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.383998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51460 (local address 127.9.133.189:32777)
0807 06:17:09.384169 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.384173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.384491 (+ 318us) server_negotiation.cc:407] Connection header received
0807 06:17:09.384671 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.384675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.384744 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.384831 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.385403 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51460: BlockingRecv error: recv got EOF from 127.0.0.1:51460 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 06:17:09.518009 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.516438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51464 (local address 127.9.133.189:32777)
0807 06:17:09.516652 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.516658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.516802 (+ 144us) server_negotiation.cc:407] Connection header received
0807 06:17:09.517045 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.517048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.517111 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.517214 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.517808 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51464: BlockingRecv error: recv got EOF from 127.0.0.1:51464 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260807 06:17:09.778292 10242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 06:17:09.776912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.9.133.189:32777)
0807 06:17:09.777090 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:09.777101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:09.777150 (+ 49us) server_negotiation.cc:407] Connection header received
0807 06:17:09.777374 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:09.777378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:09.777451 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:09.777550 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 06:17:09.778108 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51466: BlockingRecv error: recv got EOF from 127.0.0.1:51466 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 06:17:09.973897 10371 catalog_manager.cc:1259] Loaded cluster ID: 49ba8e6aa968430cbc51789423864a3c
I20260807 06:17:09.974027 10371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: loading cluster ID for follower catalog manager: success
I20260807 06:17:09.976332 10371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: acquiring CA information for follower catalog manager: success
I20260807 06:17:09.977805 10371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb7a240cc10f409aafe3866f02cbce86: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:17:10.117041 10423 catalog_manager.cc:1259] Loaded cluster ID: 49ba8e6aa968430cbc51789423864a3c
I20260807 06:17:10.117168 10423 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: loading cluster ID for follower catalog manager: success
I20260807 06:17:10.119390 10423 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: acquiring CA information for follower catalog manager: success
I20260807 06:17:10.120927 10423 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4fa8334ae385409ea5f2bd96e24d875d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 06:17:10.298342 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:10.293404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.9.133.189:32777)
0807 06:17:10.293733 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:10.293740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:10.293869 (+ 129us) server_negotiation.cc:407] Connection header received
0807 06:17:10.294129 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:10.294133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:10.294313 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:10.294484 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:10.294808 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.295435 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.296425 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.296739 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.296856 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:10.297260 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:10.297855 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:10.298023 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:10.298061 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 06:17:10.298134 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260807 06:17:10.306480 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:10.302099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.9.133.188:37161)
0807 06:17:10.302428 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:10.302432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:10.302446 (+ 14us) server_negotiation.cc:407] Connection header received
0807 06:17:10.302545 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:10.302548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:10.302693 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:10.302803 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:10.303068 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.303721 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.304610 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.304876 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.304975 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:10.305352 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:10.305929 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:10.306042 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:10.306149 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 06:17:10.306213 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":113,"threads_started":1}
I20260807 06:17:10.315140 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:10.310683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.9.133.188:37161)
0807 06:17:10.310878 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:10.310883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:10.310990 (+ 107us) server_negotiation.cc:407] Connection header received
0807 06:17:10.311167 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:10.311170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:10.311227 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:10.311303 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:10.311714 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.312513 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.313828 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.314083 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.314162 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:10.314509 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:10.314605 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:10.314696 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:10.314881 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 06:17:10.314945 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 06:17:10.315311 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:10.310114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.9.133.189:32777)
0807 06:17:10.310269 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:10.310275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:10.310876 (+ 601us) server_negotiation.cc:407] Connection header received
0807 06:17:10.311134 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:10.311139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:10.311211 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:10.311295 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:10.311775 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.312574 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.313480 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.313718 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.313835 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:10.314741 (+ 906us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:10.314822 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:10.314908 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:10.315058 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 06:17:10.315121 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 06:17:10.316581 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 06:17:10.310103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.9.133.190:35139)
0807 06:17:10.310398 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 06:17:10.310403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 06:17:10.310559 (+ 156us) server_negotiation.cc:407] Connection header received
0807 06:17:10.310828 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 06:17:10.310832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 06:17:10.311118 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 06:17:10.311285 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 06:17:10.311547 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.312292 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.313849 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 06:17:10.314162 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 06:17:10.314275 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 06:17:10.314801 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 06:17:10.315691 (+ 890us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 06:17:10.315794 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 06:17:10.316337 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 06:17:10.316429 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260807 06:17:10.320468 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10043
I20260807 06:17:10.338362 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10143
I20260807 06:17:10.354760 9750 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrsqWAA/build/debug/bin/kudu with pid 10243
2026-08-07T06:17:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3691 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7310 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (329548 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST